# KVarN

A native vLLM attention backend that quantizes the KV cache to fit three to five times more context in the same memory, without a calibration step and without changing the served model.

KVarN is a KV-cache quantization method for vLLM aimed at agentic and long-context workloads. Existing quantization approaches trade throughput for extra capacity or lose accuracy at low bit widths; KVarN is designed to keep both, matching FP16 accuracy while beating FP16 throughput and multiplying KV-cache capacity, according to results on Qwen3-32B.

It is calibration-free and plug-and-play: it adds to an existing vLLM deployment as a single flag, with no model changes. It runs in float16 compute, with a tile size equal to the vLLM block size; both a 128 (default) and a 64 tile size are supported, trading capacity for finer quantization granularity.

It ships as a vLLM fork, installed from the upstream precompiled wheel, with its kernels written in Triton and compiled just in time at runtime.

Website: https://arxiv.org/abs/2606.03458