Awesome Deep Neural Network Compression
A curated collection of papers, summaries and code on deep neural network compression, covering quantization, pruning and distillation.
Awesome Deep Neural Network Compression is a paper and code collection for neural network compression techniques, organized around quantization, pruning, both unstructured and structured, and knowledge distillation. Papers are grouped by topic, including efficient model design, neural architecture search for compression, robustness under compression, NLP-specific compression, differentiable compression, and compression for large pretraining models in language and vision.
It also lists related compression systems such as DeepSpeed, ColossalAI, Distiller and PocketFlow, and includes the maintainer's own re-implementations of compression methods and slide summaries on quantization and pruning, along with a short theory section from convex optimization to quantization.