Multi-TurboQuant
A unified KV-cache compression toolkit for LLM inference offering twelve Python-native compression methods behind one API, with guarded add-on composition, multi-GPU deployment planning and launch-command generation for llama.cpp and vLLM.
Multi-TurboQuant compresses the key-value cache that dominates memory use during large language model inference, for example an 8-plus gigabyte cache for a 32-billion-parameter model at 32K context, offering twelve compression methods plus aliases for a separate Godzilla KVarN backend behind a single API and 19 presets. A user can pick a preset and get the exact launch command for llama.cpp or vLLM with that compression enabled, or call the library directly from their own inference code, and a bundled dashboard shows detected GPUs and lets a user benchmark methods and plan deployments.
Optional heavier runtimes, such as FlashAttention, LMCache, MInference, SageAttention and a TriAttention calibrator, are managed through separate locked environments so experimenting with one add-on does not disturb the core toolkit's own dependencies, and a composition layer adds a compatibility matrix, guarded execution profiles and analytical KV-capacity estimates without itself installing or launching anything.