Awesome KV Cache Optimization
A curated, taxonomy-organized reading list of papers on system-aware KV-cache optimization for efficient large language model serving, accompanying an ACL 2026 survey.
Awesome KV Cache Optimization is a curated collection of papers on system-aware, serving-time optimization of the KV cache used by large language models, organized around a survey accepted at ACL 2026 Findings. It records methods that improve system-level metrics such as latency and memory without retraining a model or changing its architecture.
Papers are organized by a system behavior-oriented taxonomy across three dimensions: temporal, covering when the KV cache is accessed or computed, including scheduling, pipelining, and hardware-aware execution; spatial, covering where the cache is placed and migrated across the memory hierarchy or compute devices; and structural, covering how the cache is represented and managed. The survey further analyzes cross-behavior co-design affinity and behavior-objective effects to surface open research questions.
The repository accepts pull requests and issues to add new papers, and is regularly updated.