Turbovec Open-Source Vector Index Claims Substantial Memory Compression and Speed Gains Over FAISS

A new open-source vector search index called Turbovec aims to lower the hardware and memory requirements of retrieval-augmented generation (RAG) applications by implementing Google Research's TurboQuant algorithm. Written in Rust with Python bindings, the engine reduces a 10-million-document corpus from 31 gigabytes of RAM in standard 32-bit floating-point format down to 4 gigabytes, while delivering search speeds that exceed Meta’s widely used FAISS library, as first reported on Hacker News.

Designed as a drop-in replacement for standard vector and document stores across major developer frameworks, Turbovec retains standard public interface surface areas, persistence mechanics, and retriever pipeline wiring. The software targets enterprise environments where data privacy, RAM utilization, and response latency are critical, avoiding the need for data-dependent codebook training phases prior to index creation.

At the core of Turbovec is TurboQuant's data-oblivious quantization framework. The system operates by first normalizing incoming vectors into unit directions on a high-dimensional hypersphere, saving each vector's original magnitude as a single float. It then multiplies the normalized vectors by a shared random orthogonal matrix. This random rotation transforms coordinate values so that they follow a predictable mathematical Beta distribution regardless of the original underlying dataset, removing the requirement to run iterative training steps like standard k-means clustering.

For additional accuracy, Turbovec includes an optional calibration mechanism designated as TQ+. By analyzing a small representative sample of approximately 1,024 vectors prior to insertion, TQ+ calculates per-coordinate shift and scale parameters to align empirical data quantiles directly with codebook boundaries. The index then performs Lloyd-Max scalar quantization, bucketing coordinates into 4 distinct centroids for 2-bit mode or 16 centroids for 4-bit mode. This step shrinks a 1,536-dimensional float32 vector from 6,144 bytes down to 384 bytes, achieving a 16-fold reduction in footprint.

To counteract the mathematical shrinkage typical of scalar quantization, Turbovec incorporates length-renormalized scoring. During encoding, the index calculates the inner product between each rotated vector and its quantized centroid reconstruction, storing a single scalar multiplier alongside the candidate payload. The search kernel applies this scalar directly during score calculation to eliminate downward inner-product estimation bias at query time without imposing additional runtime latency or memory overhead.

In benchmark testing against FAISS IndexPQ using 100,000 vectors across 1,000 queries, Turbovec demonstrated consistent latency advantages on modern CPU architectures. On ARM processors, Turbovec averaged 3.5 times faster execution at 4-bit quantization and 26 percent faster execution at 2-bit quantization compared to FAISS FastScan. On x86 architectures, the engine delivered an average 3.4-fold speed advantage at 4-bit and a 20 percent enhancement at 2-bit, utilizing hardware-specific AVX-512, AVX2, and ARM NEON SIMD kernels.

Data mutation metrics also highlight substantial differences in index maintenance costs. Adding a single vector to a populated Turbovec index takes between 6.3 and 19.7 microseconds, operating 7.6 to 13.9 times faster than single-item additions to a trained FAISS IndexPQFastScan. For deletion operations, Turbovec performs ID removals in 0.44 to 1.22 microseconds using a constant-time swap-and-pop pattern, whereas FAISS requires 0.19 to 1.02 seconds per operation due to internal codebook repacking processes.

Turbovec also integrates selective filtering directly inside its vectorized execution loop. The search kernel evaluates candidate allowlists at a 32-vector block granularity, skipping SIMD math and memory lookups entirely for blocks that contain no eligible slots. The index outputs exact result lengths matching the lower bound of requested items or permitted candidates, offering structured execution for enterprise search pipelines constrained by complex metadata permissions.