Awesome Model Quantization
A curated collection of papers, benchmarks, surveys and open-source implementations on neural network and model quantization, spanning binary and low-bit networks, post-training and quantization-aware training, large language models, and hardware-efficient deployment.
Awesome Model Quantization collects papers, benchmarks, surveys and open-source implementations on neural network and model quantization. It covers binary and ternary networks, post-training quantization, quantization-aware training, vector and lattice quantization, low-bit large language models, multimodal and generative models, KV-cache quantization, low-precision training and hardware-efficient deployment.
It organizes the field along five dimensions: the optimization paradigm used, such as post-training quantization, quantization-aware training, quantized or parameter-efficient fine-tuning, and data-free methods; how a value is represented, from scalar to vector, lattice, binary and mixed precision; how quantization error is reduced; which tensors are quantized, such as weights, activations or KV caches; and the target models and deployment setting. Representative methods listed include GPTQ, SmoothQuant, AWQ, QLoRA and ZeroQ.