Ash Vardanian @ashvardanian Twitter profile

Pinned Tweet

Ash Vardanian

3 months

We are releasing a new tiny VLM 🎉 The model is smaller and much more accurate than our previous @unum_cloud "uform-gen" downloaded 100K times/mo The new decoder is only 0.5 billion parameters and the model is already available on @huggingface 🤗 🧵

2

17

85

Last Seen Profiles

@theMMAhomies

@SueholdenNHS

@Marsactu

@lmonart_

@MaryRVelvet_VT

@ihateomerta

@paulcuffe53

@NTRARMYOFFICIAL

@Weneseo1

@Woodrow17165268

@MariciaLyn

@TBarbarroja

@NutriBrain_ct

@MargareteRosali

@amankhanbabuzai

@Waynelovescats

@WhereSmut

@MyLorals

@RSSorg

@Tsuki_Seiba

@zoepancakes_

@Maz__Tess

@Shiran_490

@GRT_ROW

@RnS2y

@YOGB_Mate

@Ohthats___tee

@NakazawaTatsuya

@TaiwanNewsEN

@MileiConduccion

@brian_blum1

@Worship_Kittyx

@arakintou

@Nene_yr

@bestjjaehever

@deyabo0

Ash Vardanian

@ashvardanian

6 months

Cosine Similarity: - 2'500x Faster than 🐍 - 12x faster than C I've combined AVX-512 VNNI extensions with BMI2 and masked loads, all of those seem to be surprisingly rare in open-source

Python, C, Assembly - 2'500x Faster Cosine Similarity 📐

In this fourth article of the “Less Slow” series, I’m accelerating Unum’s open-source Vector Search primitives used by some great database and cloud providers to replace Meta’s FAISS and scale-up...

ashvardanian.com

5

11

132

Ash Vardanian

@ashvardanian

6 months

I may be wrong here, but is my open-source library 1000x more cost-efficient than a closed-source product that has recently raised over $100M?

4

12

74

Ash Vardanian

@ashvardanian

1 year

We are open-sourcing UForm, a Multi-Modal inference lib for Semantic Search! It extends @huggingface transformers with Mid-Fusion and comes with a light-weight multi-lingual pre-trained Vision-Language model, beating @OpenAI CLIP in speed and accuracy 🥳

GitHub - unum-cloud/uform: Pocket-Sized Multimodal AI for content understanding and generation...

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️ - unum-cloud/uform

github.com

2

12

68

Ash Vardanian

@ashvardanian

2 months

StringZilla is trending on @github 🎉 The world wastes at least $100M a year due to low CPU utilization in basic string operations - let's fix them together 🤗

1

7

58

Ash Vardanian

@ashvardanian

6 months

Billion Scale Vector Search on 1 server. Over 200,000 queries/second. is probably half of that throughput.

2

4

56

Ash Vardanian

@ashvardanian

1 month

Indexing and searching 40 Million English Wikipedia pages on a laptop? 13'000 insertions/second 26'000 queries/second😎 Recipe: pull binary @cohere text embeddings from @huggingface , take numerics and search from @unum_cloud , and a laptop with 32 GB of RAM 🧵

2

4

57

Ash Vardanian

@ashvardanian

10 months

Thanks for sharing! Stealing this diagram for future talks 😅

Mark Callaghan

@MarkCallaghanDB

10 months

Yet another great LeanStore paper

1

16

84

1

12

57

Ash Vardanian

@ashvardanian

6 months

Big News 📢 I've partnered with @awscloud Open Data to release 28 billion molecule embeddings - potentially the largest public #cheminformatics dataset, searchable with @unum_cloud USearch, to accelerate AI-powered drug discovery and broader science ⚗️

6

14

53

Ash Vardanian

@ashvardanian

1 year

Last week was 🔥 for vector search. Weaviate raised $50M, and Pinecone raised $100M... That's a lot and makes you believe that vector search is hard. But it's not. I have spent the last few days implementing a single-file vector search engine... 🧵 1/7

Unum

@unum_cloud

1 year

We are excited to share a single-header C++11 vector-search engine with SDKs for #Python , #JavaScript , #Java , #Rust , and soon #GoLang & #Wolfram ! It's tiny and easy to use, but still brings half-precision & integer quantization and scales beyond 4B points!

1

7

35

2

5

44

Ash Vardanian

@ashvardanian

7 months

I’ve found a fairly simple data-parallel workload, where AVX-512FP16 #SIMD code beats O3/fast-math assembly produced by GCC 12 by a factor of 118x 🤯🤯🤯

118x faster than GCC 12: KL & JS divergence with AVX-512FP16 by ashvardanian · Pull Request #34 ·...

Divergence functions are a bit more complex than the Cosine Similarity, primarily because they have to compute logarithms, which are relatively slow when using LibC's logf. So, aside from minor...

github.com

4

8

41

Ash Vardanian

@ashvardanian

5 months

If you need a hardware-accelerated Vector Search on iOS devices - check out USearch 😉 - already used by publishers with 1M+ installations, here is a Swift demo app:

GitHub - ashvardanian/SwiftSemanticSearch: S3 stands for Swift Semantic Search - using USearch &...

S3 stands for Swift Semantic Search - using USearch & UForm to builds scalable multimodal Retrieval-Augmented pipelines on Apple devices 🍏 - ashvardanian/SwiftSemanticSearch

github.com

Robert Lukoszko — e/acc

@Karmedge

5 months

Apple has started showing/leaking some of their LLM developments in public. during last few weeks I saw this: - WebGPU now available for testing in Safari Technology Preview - "MLX" – array framework for Apple Silicon - "LLM in a flash" paper to run efficiently on device -…

8

42

251

1

6

40

Ash Vardanian

@ashvardanian

3 months

UForm-Gen free #opensource ~1.5B parameter model vs Google Gemini, who'd win? Already 70k downloads from @huggingface this month! Definitely a cherry pick on our side, but I'm still proud for @unum_cloud and excited for next models Spotted by @or_vov 🤗

SkalskiP

@skalskip92

3 months

Gemini Advanced vs. GPT-4V? I did some tests, and Gemini Advanced is behind other LMMs. At least when it comes to image processing. - Only one question about the image. - No multi-image reasoning. - “Sorry, I can't help with images of people yet.” ↓ Read more

20

18

148

1

6

36

Ash Vardanian

@ashvardanian

2 months

CUDA and C++ Hackers gathering at @AGIHouseSF today 🚀🚀🚀 Thanks to @khoomeik , @kylejohnmorris and @JvNixon for hosting 🤗 Should we do it every month?

5

33

Ash Vardanian

@ashvardanian

3 months

@CramerTracker Nobody has to choose between a CPU and a GPU - everyone should get both 😅

3

0

34

Ash Vardanian

@ashvardanian

1 month

This is no joke - the most portable approximate vector search engine is getting even more portable. Over 100M devices reached already 😎 Thanks to @ngalstyan4 , @sroussey , and @simonw 🤗

2

31

Ash Vardanian

@ashvardanian

10 months

@LukeGessler This can be improved by replacing GZip with different compression, and can be easily combined with something like HNSW to perform log-time-search. With USearch you can also store texts already compressed, and decompress+recompress on the fly while computing distances 😉

1

30

Ash Vardanian

@ashvardanian

28 days

This week StringZilla 🦖 received a new much friendlier mascot and improved 🦀 Rust support! And if it wasn’t enough to start using it, Google is also switching to Arm-based CPUs for cloud. 50% faster search than LibC on x86 250% faster on Arm 💪

GitHub - ashvardanian/StringZilla: Up to 10x faster strings for C, C++, Python, Rust, and Swift,...

Up to 10x faster strings for C, C++, Python, Rust, and Swift, leveraging SWAR and SIMD on Arm Neon and x86 AVX2 & AVX-512-capable chips to accelerate search, sort, edit distances, alignment...

github.com

2

30

Ash Vardanian

@ashvardanian

1 month

SimSIMD crosses 100 kernels and 300K PyPi downloads 🎉 The C 99 core also reaches 50K git-clones per month, but what about Rust and JavaScript bindings? I guess computing dot-products and cosine distances should be universally important across languages 🤷‍♂️

2

1

26

Ash Vardanian

@ashvardanian

4 months

Just a couple of hours since the @unum_cloud /uform-gen release, and the community already created a @huggingface gradio example for image captioning🔥 👏 @albfresco

Ugen Image Captioning - a Hugging Face Space by aavetis

huggingface.co

2

8

25

Ash Vardanian

@ashvardanian

1 year

Huge congrats to our fellow Armenian colleagues at @Picsart , currently ranking first on HackerNews with their recently released text-to-video models!

0

3

27

Ash Vardanian

@ashvardanian

3 months

USearch v3.9 brings SQLite extensions for Vector, Full-Text, and GIS Search 🎉 It can handle BLOBs, TEXTs, JSONs, bitsets, and arbitrary scalar columns analyzed together. It exposes all kinds of distances from Cosine to Levenshtein - from measuring the distance between AI model…

4

27

Ash Vardanian

@ashvardanian

3 months

Vector Search in @ClickHouse is getting faster and faster - from 33 seconds to 0.064 seconds for 33 million 768d vectors.. is it a 515x performance improvement?! 🤯 Proud to see such applications of our tech - @unum_cloud USearch🚀

3

0

27

Ash Vardanian

@ashvardanian

5 months

@jimkxa @whatifalthist ... Space exploration? Gravitational telescopes?

0

24

Ash Vardanian

@ashvardanian

2 months

Microbenchmarks are fine, but what about real-world applications? I've patched @huggingface datasets library common in AI training pipelines to use StringZilla's SIMD-based iterators. Resulted in 3x throughput increase, so probably worth pushing to upstream🤔

1

4

25

Ash Vardanian

@ashvardanian

8 months

USearch v2.6 is out with ~3-6x faster than FAISS exact search, even without my SimSIMD lib 🤯 Pros, what speed difference in construction and search would you anticipate for a 1 Billion points HNSW index in USearch vs FAISS? 10x? 100x?

3

2

24

Ash Vardanian

@ashvardanian

11 days

What a day it was! While @peshotrie was presenting his work in Boston at RECOMB 2024, our team was accelerating DNA sequence alignment in San Francisco 🌁 We've patched 3 #opensource libraries and published a new dataset 🤯 Let's hope it accelerates pharma R&D and we all get to…

AGI House SF

@AGIHouseSF

11 days

@khoomeik DNA-Zilla - accelerating DNA sequence matching with A* heuristics Overheard some of their conversations…really interesting hardcore algorithms work and all shipped open source @ashvardanian @alexbowe @Adriano34554795

1

6

1

0

24

Ash Vardanian

@ashvardanian

4 months

Wow! People are actually using UCall (70x faster FastAPI) and it seems to work fine - over 100K downloads already! The main branch is over 100 commits behind and quite outdated, but seems like I must find a way to grow our team to support all these 😅

Feature: Python 3.12 support · Issue #92 · unum-cloud/ucall

Describe what you are looking for Python 3.11 support is working great but, ecosystem moves to Python 3.12. Can you support Python 3.12 and publish new version? Can you contribute to the implementa...

github.com

1

4

24

Ash Vardanian

@ashvardanian

3 months

We also now have a much more complete @Gradio space to browse different examples and play with the model 🤗

Ash Vardanian

@ashvardanian

3 months

We are releasing a new tiny VLM 🎉 The model is smaller and much more accurate than our previous @unum_cloud "uform-gen" downloaded 100K times/mo The new decoder is only 0.5 billion parameters and the model is already available on @huggingface 🤗 🧵

2

17

85

0

3

21

Ash Vardanian

@ashvardanian

2 months

I was waiting for @AMD Bergamo chips for many years… But this Genoa-X variant sounds insane. With over 1 GB of L3 cache and 192 threads, you can literally fit an entire Alpine Linux Docker image on every vCPU… with every image cached on the CPU itself… not in RAM 🤯

Ash Vardanian

@ashvardanian

2 months

Choosing between Intel and @AMD CPUs? Considering AVX512 support and cost-efficiency? Intel first implemented AVX512 in 2016. AMD recently added those in Zen4 CPUs you can get in AWS "7a" instances. So I've benchmarked several SIMD kernels and here are my first observations 🧵

1

7

1

21

Ash Vardanian

@ashvardanian

5 months

Oh, nice! SimSIMD-related article is trending on the first page of HackerNews 🎉

1

2

22

Ash Vardanian

@ashvardanian

5 months

Google has published a competitive neural network with less than 1 million parameters! It's designed for efficient large-scale deduplication and similarity search, was published with a new multi-lingual W4NT3D benchmark for text retrieval in an adversarial setting, and uses…

0

3

20

Ash Vardanian

@ashvardanian

25 days

Apple has over 2 billion devices worldwide. Most of them are equipped with powerful GPUs. Time to bring multimodal AI to every pocket. Time to bring UForm to 🍏

GitHub - unum-cloud/uform: Pocket-Sized Multimodal AI for content understanding and generation...

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️ - unum-cloud/uform

github.com

Alex Reibman 🖇️

@AlexReibman

25 days

12/ UForm 4 Swift Family of pocket-sized multimodal AI models (small enough to run on watches) re-written in ONNX and ported to Swift @ashvardanian

1

4

14

3

5

20

Ash Vardanian

@ashvardanian

5 months

@timur_audio Developer experience. Haven't tried vcpkg, but submitting to Conan is an unintuitive manual process. For PyPi and NPM it takes me just 1 CI action to upload a release: CMake fetch is almost ok for header-only stuff, but I'll never get used to its syntax

0

20

Ash Vardanian

@ashvardanian

8 months

Typical “build it and they will come” scenario: 🦖 #opensource #python #substring #search

1

3

19

Ash Vardanian

@ashvardanian

3 months

What a spike in downloads yesterday?! @unum_cloud UForm-Gen multimodal transformer crosses 15K downloads from @huggingface in the first month 🥳 In the meantime, we have 5 other different-size multimodal models in the training - it's gonna be a very packed February!

1

17

Ash Vardanian

@ashvardanian

7 months

Do you use #NumPy or #SciPy to compute vector similarity? For @OpenAI Ada embeddings – SimSIMD does it 3-200x faster 🤯 float16 works great on every arch - Arm NEON & SVE and x86 AVX2 & AVX-512. Intel AMX also looks excellent 🔜 #opensource #Python

GitHub - ashvardanian/SimSIMD: Up to 200x Faster Inner Products and Vector Similarity — for Python,...

Up to 200x Faster Inner Products and Vector Similarity — for Python, JavaScript, Rust, and C, supporting f64, f32, f16 real & complex, i8, and binary vectors using SIMD for both x86 AVX2 &a...

github.com

3

2

16

Ash Vardanian

@ashvardanian

4 months

StringZilla v3 is starting to look serious! Crushing C++ standard library by 3-20x is one thing... But beating LibC on real-world data is what really what made my morning 🔥 The development branch:

1

2

18

Ash Vardanian

@ashvardanian

5 months

Here is a simple example showcasing the cost of `if` statements in code. Up to 4096 branches - all memorized, no branching happens and the expression evaluates in only 2 CPU cycles. After that on average ~10 cycles are wasted per iteration. 5x slowdown.

Ash Vardanian

@ashvardanian

5 months

HPC gurus, I'm harvesting short code snippets that would trouble the CPU's branch predictor. Any suggestions? This variant only adds around 5 cycles on average:

1

6

1

0

18

Ash Vardanian

@ashvardanian

2 years

We went through life with a smile. Now I am smiling through tears, alone. Yesterday was the memorial service. Yesterday I was sitting next to a coffin, with the love of my life and my child in it. I must tell their sorry.

2

3

18

Ash Vardanian

@ashvardanian

1 year

With this new website theme my old “Mastering C++ with Google Benchmark” beginner tutorial looks better than ever! Especially the code highlighting and the ability to reference specific lines in Markdown code sections!

Mastering C++ with Google Benchmark ⏱️

There are only two kinds of languages: the ones people complain about and the ones nobody uses. – Bjarne Stroustrup, creator of C++. Very few consider C++ attractive, and only some people think it’s...

ashvardanian.com

3

2

17

Ash Vardanian

@ashvardanian

3 months

Compared to the C++ standard library on @Arm : - 4.4x faster substring search - 16.8x faster in reverse order - 1.7x faster sorting + vastly more functionality! Are you using @awscloud Graviton, @AmpereComputing Altra, @Azure Cobalt, @Nvidia Grace... You need StringZilla 😉

Ash Vardanian

@ashvardanian

3 months

Bindings for C++, Rust, Swift, AVX-512, Levenshtein Distance & Needleman-Wunsch Scores, Rolling Fingerprints, Lazy Ranges, Faster Sorting... Yes, StringZilla v3 is out! 🦖3️⃣🎉 Special thanks to @keithmadams , @vatsal_manot , & @MikayelGrigory3 🤗

0

5

15

0

3

17

Ash Vardanian

@ashvardanian

2 months

New post - NumPy vs BLAS: Losing 90% of Throughput I've compared the performance of NumPy against it's underlying OpenBLAS implementation, recording up to 10x throughput difference in dot-products of 1536-dimensional vectors, that @OpenAI Ada produces

NumPy vs BLAS: Losing 90% of Throughput

Downloaded over 5 Billion times, NumPy is the most popular library for numerical computing in Python. It wraps low-level HPC libraries like BLAS and LAPACK, providing a high-level interface for...

ashvardanian.com

1

3

16

Ash Vardanian

@ashvardanian

3 months

Beating standard Linux utilities - 3x faster `wc`, 4x faster `split` 😎 I've used StringZilla Python SDK to prototype replacements for popular CLI tools that deal with large texts. Initial results look great. Any Python and Shell lovers willing to join?

3

2

16

Ash Vardanian

@ashvardanian

4 months

Yes, it was supposed to be about the company and its story, but I still managed to bring up SIMD, and how we use Arm SVE to go beyond "fast SIMD libraries" with SSE support and compiler auto-vectorization pragmas... no cure for me 😅 Thanks to the amazing @cerebral_valley team!

Cerebral Valley

@cerebral_valley

4 months

We're live with our first CV Deep Dive, featuring @ashvardanian and @unum_cloud : "How USearch, @ashvardanian 's Vector-Search Engine, Reached 500k+ Python Downloads 🌐" Ash covers OSS, multi-modal and more... Share and RT if you found it valuable 👀 Link below 👇

1

2

11

0

3

15

Ash Vardanian

@ashvardanian

1 month

The amazing world of Python. Load 2 GB of floats - 77 seconds. Downcast them to bits - 53 seconds. The frustration - priceless. For everything else there is MasterCard...

0

16

Ash Vardanian

@ashvardanian

2 months

OMG! TensorFlow and JAX call latency is insane, totally inappropriate for small vector-vector operations. JAX is 6x faster than TF, Torch is 10x faster than JAX, NumPy is 10x faster than Torch, And SimSIMD is another 2x faster than NumPy 😎

3

6

16

Ash Vardanian

@ashvardanian

6 months

USearch just cross 1000 ⭐️ and SimSIMD is turning 3 months old 🎉 Thanks to everyone for upvotes and spreading the word! With major releases in @unum_cloud UCall and StringZilla coming this December - its my biggest year in opensource since registering on #github in 2012

1

2

16

Ash Vardanian

@ashvardanian

6 months

It blows my mind, that even on `double`-precision floating point numbers NumPy and SciPy (using OpenBLAS) are so slow! Cosine distance in SimSIMD: - 9x faster on Intel Sapphire Rapids (thx to AVX-512) - 18x faster on AWS Graviton 3 (thx to SVE)

Release Faster Double-Precision Math · ashvardanian/SimSIMD

As was discussed in the SciPy integration thread, Python libraries use double-precision floating-point numbers by default. So in this release I've extended the spatial distance functions - cosi...

github.com

1

0

16

Ash Vardanian

@ashvardanian

2 years

My talk was on the basics of using Google Benchmarks with examples of misaligned memory loads, C++ attributes, ffast math and more!

Ash Vardanian

@ashvardanian

2 years

Yesterday we had an amazing monthly gathering at C++🇦🇲 with talks exclusively on benchmarking, both from @Siemens and @unum_cloud 100+ attendees, cinema hall and 🍿popcorn, what else can one desire? 😂 #cpp #meetup

2

8

1

3

15

Ash Vardanian

@ashvardanian

6 months

Two things I didn't know about #FAISS : 1. It also supports clustering and down-casting to f16/i8. 2. It has good performance at start, but as you grow datasets - it plummets. USearch is easily 10x faster at scale. also 10x smaller binary. and has bindings to 10 languages 😎

Ash Vardanian

@ashvardanian

6 months

Billion Scale Vector Search on 1 server. Over 200,000 queries/second. is probably half of that throughput.

2

4

56

2

3

15

Ash Vardanian

@ashvardanian

7 months

USearch - the fastest Vector Search engine crossing 300,000 #Python downloads 🎉 Assuming the AI wave is about to hit every software developer, there are also bindings for JS, Rust, Go, Java, ObjC, Swift, C#, etc. Just need to find a way to popularize them 🤦‍♂️

1

0

15

Ash Vardanian

@ashvardanian

3 months

Bindings for C++, Rust, Swift, AVX-512, Levenshtein Distance & Needleman-Wunsch Scores, Rolling Fingerprints, Lazy Ranges, Faster Sorting... Yes, StringZilla v3 is out! 🦖3️⃣🎉 Special thanks to @keithmadams , @vatsal_manot , & @MikayelGrigory3 🤗

0

5

15

Ash Vardanian

@ashvardanian

1 month

Microsoft, Salesforce, and @unum_cloud are behind some of the @huggingface 's most trending multimodal models. David and Goliath, AI edition 2024 💪

1

3

15

Ash Vardanian

@ashvardanian

10 days

Together with @vatsal_manot we've assembled a demo showing how to run real-time multimodal AI pipelines on @Apple devices in Swift using @unum_cloud and @PreternaturalAI Zero API calls, 100% privacy Check out the demos ⤵️

GitHub - ashvardanian/SwiftSemanticSearch: S3 stands for Swift Semantic Search - using USearch &...

S3 stands for Swift Semantic Search - using USearch & UForm to builds scalable multimodal Retrieval-Augmented pipelines on Apple devices 🍏 - ashvardanian/SwiftSemanticSearch

github.com

1

3

17

Ash Vardanian

@ashvardanian

10 days

I didn't realize ~42% of M3 Max area is a GPU! With 92 billion transistors in a chip, the GPU part is comparable to a 4090 Sadly, our UForm models can't yet access the NPUs for matrix multiplications, and I am curious if there is any piece of software using AMX. Seems redundant

0

15

Ash Vardanian

@ashvardanian

5 months

I've made Great Circle distance computations 4x faster compared to LibC-based on Arm Not a typical SIMD case, only 4 floats & 3 trig. functions Working on hybrid GIS/Vector/Text search in USearch now

1

2

14

Ash Vardanian

@ashvardanian

5 months

@Karmedge There has been a ton of fuzz around MLX, but it seems to be a vanilla high-level wrapper around their DSP SDK and Metal API. My expectations were much higher If you are really interested in doing serious compute on Apple devices, check this:

GitHub - corsix/amx: Apple AMX Instruction Set

Apple AMX Instruction Set. Contribute to corsix/amx development by creating an account on GitHub.

github.com

1

14

Ash Vardanian

@ashvardanian

1 month

USearch has had filtering support (predicate callbacks) in its low-level engine for a long time. Now, I am exposing them to C and Rust APIs 🥳 PR: PS: ChatGPT and other LLMs are so useless when it comes to well documented behaviors, that aren't used by…

Richer High-Level APIs for Rust 🦀 and C 🐂 by ashvardanian · Pull Request #388 · unum-cloud/usearch

In C 🐂 usearch_filtered_search with stateful predicates usearch_change_metric usearch_change_metric_kind usearch_change_expansion_add usearch_change_expansion_search improved docs In Rust 🦀...

github.com

0

1

14

Ash Vardanian

@ashvardanian

4 months

StringZilla is getting Rust and Swift bindings, as well as dynamic runtime dispatch packaged into a tiny C library without the LibC dependency 🎉 More benchmarks coming!

0

3

13

Ash Vardanian

@ashvardanian

3 months

Systems Hackathon Alert 🎉 February 24th, San Francisco. Full day gathering for those who build AI training frameworks, compilers, databases, search engines, and other infra! No LLM wrappers this time 😉 Co-organized by the @AGIHouseSF and @CppBayArea

Hardcore Hackathon @ AGI House SF | Partiful

Calling all CUDA, C/C++, Rust, Assembly, systems, networking, & low-level hackers! This hackathon will be a more intimate & high talent density gathering than our usual events, exclusive to systems...

partiful.com

1

4

14

Ash Vardanian

@ashvardanian

3 months

I've benchmarked one of most popular #Rust libraries `memchr` (200M downloads) against `stringzilla` 🦖 Won 7 benchmarks out 8. I was hoping for a clear win, but apparently I still have things to optimize on Arm 💪 The benchmarks are very easy to repeat:

2

0

14

Ash Vardanian

@ashvardanian

1 month

This is sick. A decade has passed since that post, but it doesn't seem like we've made much positive progress. @lemire has a great point as usual 🤗 So if you are by any chance using my "usearch-molecules" dataset for drug discovery and broader biotech (or any other of my open…

Daniel Lemire

@lemire

1 month

@ashvardanian Do we need patents?

0

1

5

0

3

13

Ash Vardanian

@ashvardanian

11 months

USearch got a new release! I recently learned about @MaratDukhan ’s PeachPy project an immediately applied it to produce x86 Assembly directly in the Python layer and inject it into the existing binary! #Arm and #RISCV next!

Release Scaling Vector Search: Numba, Cppyy, PeachPy, and SimSIMD · unum-cloud/usearch

USearch has now been used in two internet-scale companies/projects 🎉 In one case, it's only 1 Billion vectors. The second is closer to 10 Billion per node. So it feels like a good time to enter...

github.com

0

1

14

Ash Vardanian

@ashvardanian

1 year

New phase for @unum_cloud ! After years of building, we start integrations and UForm is just the first step. Outside of Big Tech we are the first ones with end-to-end optimized Embeddings, Databases, and Vector Search, all #opensource !

Unum

Exascale Search & AI. Unum has 25 repositories available. Follow their code on GitHub.

github.com

SimFog

@FogSim

1 year

New GPTCache's Feature: Support the uform embedding, which can be used the bilingual (english + chinese) language thanks @ashvardanian 's contribution !!!

0

1

5

0

1

8

Ash Vardanian

@ashvardanian

4 months

Good news for AI infra engineers! LLVM now provides runtime functionality to detect if an x86 CPU supports SIMD extensions for IEEE-compliant half-precision floats

Lacking `f16c` and `avx512fp16` CPU feature strings for half-precision SIMDs · Issue #65320 ·...

GCC conveniently provides the __builtin_cpu_supports("f16c") and __builtin_cpu_supports("avx512fp16") used here, to detect the availability of half-precision float16 SIMD-instru...

github.com

0

13

Ash Vardanian

@ashvardanian

1 month

The appetite for SIMD is growing. This slide is from 8 months ago, from a talk that was just made public. StringZilla ⭐️ 22 -> 1.7K = 70x SimSIMD⭐️ 15 -> 0.7K = 45x USearch ⭐️0.3K -> 1.5K = 5x Over 100 Million devices reached 🥳

2

0

13

Ash Vardanian

@ashvardanian

3 months

There is an extremely timely article by @lemire on rolling hashes! I'm having a hard time going beyond 1 GB/s/core for 64-bit hashes in StringZilla v3. I've shared a few links under the HN discussion and would definitely appreciate advice 🤗

Daniel Lemire

@lemire

3 months

How fast is rolling Karp-Rabin hashing? A hash function maps values (e.g., strings) into a fixed number of strings, typically smaller than the original. It is useful to compare quickly two long strings, for example. Instead of comparing the strings, you may compare the hash…

5

15

114

4

1

13

Ash Vardanian

@ashvardanian

4 months

I love the smell of new AI models baking 🧠

1

0

13

Ash Vardanian

@ashvardanian

4 months

Apple seems to be producing the best consumer grade chips for AI. We've benchmarked UForm with @BlackForestBoi on RTX 3090 vs two M2 CPUs, and UForm on M2 crushed LLaVA on GPU even in raw throughput, let alone efficiency🤯Anyone with M3 and RTX 4090?

2

1

13

Ash Vardanian

@ashvardanian

4 months

USearch just crossed 500,000 Python downloads! Enjoy the holidays and join the @unum_cloud AMA on Thursday 10 am 🎉

3

1

12

Ash Vardanian

@ashvardanian

13 days

I've said it before and will repeat again. More conferences should start with a poker tournament. Kudos to @openoceanvc 🤗

1

0

12

Ash Vardanian

@ashvardanian

6 months

In the 3 most popular open-source projects I maintain, the next major releases are full rewrites with extended functionality 🤯 Its one of those situations when you are equally excited and scared, esp trying to fit >1M LOC functionality into less than 100K LOC of C99 😰

0

12

Ash Vardanian

@ashvardanian

7 months

I have been on LinkedIn for a few years now, and it seems like they've got the ChatGPT preview long before the inception of OpenAI 😅

Alex Ker 🔭

@thealexker

7 months

LinkedIn is about to get a whole lot better 🌤️:

40

27

336

0

12

Ash Vardanian

@ashvardanian

1 year

@RReverser In C for such verbose interfaces we simply use structs as arguments. Should be easy adopt the same pattern in any high-level language:

0

12

Ash Vardanian

@ashvardanian

6 months

I wrote a script and computed the stargazers intersection between some of the most popular projects on @github , across databases, web, AI, and basic utilities including some of my own 😅 1/11🧵

Ash Vardanian

@ashvardanian

6 months

So the question is, how many of those 877 people intersect?

0

8

4

2

12

Ash Vardanian

@ashvardanian

2 months

My Linux Foundation 30m talk on large-scale RAG, search, and small Language Models is live on Youtube, but here some of my favorite parts with broad implications for entire industry 🧵

Retrieval Augmentation and Semantic Search at Scale - Ash Vardanian,...

Retrieval Augmentation and Semantic Search at Scale - Ash Vardanian, Unum CloudThe topic of Vector Search and Retrieval-Augmented Generation has gained much ...

www.youtube.com

2

3

12

Ash Vardanian

@ashvardanian

3 months

Wow. The new 30x improved pg_vector indexes 1 million vectors in ~10 minutes... The same operation in @unum_cloud USearch can be done in ~10 seconds... another at least 60x space for optimization in @supabase

Paul Copplestone — e/postgres

@kiwicopple

3 months

@pgvector v0.6.0 was just released. Previously building HNSW indexes took a long time. This release fixes that we benchmarked it against the v0.5.1: it's not just a small improvement - it's up to 30x faster coming soon to @supabase . benchmarks in thread ↓

7

13

86

4

0

12

Ash Vardanian

@ashvardanian

2 months

Done, SimSIMD dynamic SIMD dispatch now also works on Windows starting with v4.1 🤗 🪟 USearch next!

Release SIMD on Windows 🤗 🪟 · ashvardanian/SimSIMD

This release refactors compiler attributes and intrinsics usage to make it compatible with MSVC. Most noticeably, function defined like this: __attribute__((target("+simd"))) inline stati...

github.com

Ash Vardanian

@ashvardanian

2 months

Weird problem of the day. MSVC lacks the `_mm_rsqrt14_ps` reciprocal square-root intrinsic, but has its masked variant - `_mm_maskz_rsqrt14_ps`. Why?! That is to say, SimSIMD and USearch are about to get dynamic SIMD-kernel dispatch on Windows as well 🤗🪟

2

0

3

0

2

12

Ash Vardanian

@ashvardanian

27 days

A Freudian Slip in San Francisco The weird things you can find at @cerebral_valley HQ 😅

0

11

Ash Vardanian

@ashvardanian

6 months

Thanks to @ZFPhalanx USearch just crossed 1,000 stars and 90,000 monthly PyPi downloads 🎉🎉🎉 I’m very excited about the next major release combining it with StringZilla for SIMD-accelerated hybrid vector and text search

phalanx

@ZFPhalanx

6 months

ベクトル検索エンジンusearch クソ早い

1

80

639

1

11

Ash Vardanian

@ashvardanian

3 months

There are several specialized Levenshtein distance computation libraries in the Python ecosystem. Practically all of them are implemented in C, same as StringZilla... Even without the upcoming assembly optimizations it's already 2-7x faster 🫳🎤

0

1

11

Ash Vardanian

@ashvardanian

1 year

@lemire Sounds good, don't you think?

1

0

11

Ash Vardanian

@ashvardanian

2 years

C++🇦🇲 group is rapidly growing! Mostly <35yo, 30% women, with lots of members from @synopsys @Siemens @XilinxInc @krispHQ and @unum_cloud , of course! Maybe time to follow @corecpp and @cpp_russia and organize a local conf? #cpp #programming #community

1

3

11

Ash Vardanian

@ashvardanian

8 months

@jedisct1 Oh, you must pay if you want to see those expected particles expose themselves 😂

0

2

11

Ash Vardanian

@ashvardanian

2 months

Anyone working with big data? I'm merging a huge PR, adding new APIs to StringZilla for lazy dataset processing. In some cases it results in 10x throughput improvement at 20x lower memory consumption even for `split`. Using the new `split_iter` the mem. usage approaches zero 🧵

3

0

11

Ash Vardanian

@ashvardanian

1 month

Unikraft is launching KraftCloud - low latency cloud platform build around unikernels - tiny single-purpose bootable application images 🔥

Introducing KraftCloud

At Unikraft we're introducing KraftCloud a next generation cloud platform powered by technology intended to work in millisecond timescales.

unikraft.io

1

4

11

Ash Vardanian

@ashvardanian

6 months

The new documentation is looking good! I am still waiting for @github to allow HTML tabs, so that we fit the QuickStart guide for all 11 programming languages on 1 page.

1

0

10

Ash Vardanian

@ashvardanian

1 month

Get ready for the next Hardcore Systems Hackathon in San Francisco! C, C++, CUDA, Rust, and other software and hardware engineers, let's gather together at the @AGIHouseSF on May 12th to work on computing infrastructure together 🤗

RSVP to Hardcore Systems Hackathon | Partiful

Calling all CUDA, C/C++, Rust, Assembly, systems, networking, & low-level hackers! Come join us, together with Alexander Gallego (Founder of RedPanda), Nikita Shamgunov (Founder of Neon & Partner at...

partiful.com

0

3

11

Ash Vardanian

@ashvardanian

2 months

... the perennial programming quandary: "How do I take good data in string form and painlessly turn it into garbage?" Source: GNU C Library How have I survived without ever using this 😂

1

10

Ash Vardanian

@ashvardanian

4 months

SimSIMD is on the HackerNews main page - probably for the first time 🎉 Thanks to @sroussey for porting it to every JavaScript runtime and spreading the word - go get "hardware-accelerated similarity metrics" from NPM for your LLM applications 🤗 HN:

1

10

Ash Vardanian

@ashvardanian

2 years

Even at war, we stay compassionate to those looking for shelter. Even losing our children, we find the strength to fight, to live, and to love. I am proud to be #Armenian 🇦🇲

0

1

10

Ash Vardanian

@ashvardanian

4 months

I was attempting to use ternary logic x86 instructions in StringZilla today and failed miserably... but discovered these few macros with precomputed masks on GitHub, in case you ever need them @pshufb also has a project that would help you generate such masks

1

2

10

Ash Vardanian

@ashvardanian

2 years

Best CPU vs Best GPU: 50x NumPy = CuPy! @unum_cloud @ashvardanian #cuda #python #numpy #gpu

1

2

10

Ash Vardanian

@ashvardanian

4 months

StringZilla C++ binding, replacing the standard <string> is already 3,500 lines of code 🧠🔫 That’s excluding the C-level SIMD implementation, which is expected to cross 4,500 with NEON and AVX2 added back. Anyone wants to add more test cases:

1

10

Ash Vardanian

@ashvardanian

21 days

7'000 GitHub stars reached across StringZilla, SimSIMD, and @unum_cloud USearch, UForm, UCall 🎉 Thanks to all the amazing contributors! It's sometimes hard to believe that others would be ready to invest their free time to refactor my crappy CMake scripts 😅

0

10

Ash Vardanian

@ashvardanian

5 months

Jeff Bezos on the @lexfridman podcast ep. 405 - I know what I’m watching tonight 🎉

Jeff Bezos: Amazon and Blue Origin | Lex Fridman Podcast #405

Jeff Bezos is the founder of Amazon and Blue Origin.Thank you for listening ❤ Please support this podcast by checking out our sponsors:- Notion: https://noti...

www.youtube.com

1

0

9

Ash Vardanian

@ashvardanian

2 months

What's wrong with AI for Science?! I've looked into 3 papers in the last hour. First model was on Google Drive, the second was on Microsoft OneDrive, and the third was on IBM Box 🤯 How about Git LFS? S3 & object stores? Or @huggingface if you aren't into reinventing wheels?

1

0

9

Ash Vardanian

@ashvardanian

2 months

@lin72h We aim to support all modern compilers - GCC, LLVM, and MSVC. LLVM cross-compilation works fine and is used extensively already for our C and C++ code. Zig is a nice language, but we don't really need it.

0

1

Ash Vardanian

@ashvardanian

1 year

It was fun, and to my surprise, it performed really well, reaching 300K QPS on Amazon c7g instances. I had never used third-party vector search products, but the first testers of #USearch suggested 3x performance improvement over their existing solutions 🔞 🧵 5/7

1

0

9