TurboQuant Pro
A compression library for embedding indexes and LLM KV caches that optimizes for the metric a downstream consumer actually uses, such as retrieval recall or generation quality, instead of reconstruction error alone.
TurboQuant Pro compresses vectors in embedding indexes and language model KV caches by the metric that matters to their downstream use: retrieval recall for indexes, and attention or generation quality for KV caches. Its authors argue that the common approach of optimizing reconstruction cosine similarity is a poor proxy for either of these, and can even be misleading.
The package includes architecture-aware key quantization for KV caches, compressed-domain search, sharded and memory-mapped indexes larger than RAM, drift monitoring, and a replay command that reproduces its published benchmark claims. It installs with pip and reports large compression ratios with near-lossless recall on public embedding benchmarks, alongside a claims file that tracks the reproduction status of every reported number.