LightCompress
A toolkit for compressing large generative AI models, including language models, vision-language models and video-generation models, using quantization, pruning and token-reduction algorithms to cut size while preserving performance.
LightCompress, formerly known as LLMC, is a toolkit for compressing large AIGC models such as LLMs, VLMs and diffusion and video-generation models. It applies compression algorithms, including quantization and pruning, aiming to reduce model size and improve deployment efficiency without a significant drop in performance.
It supports over 20 algorithms covering both token reduction and quantization for vision-language models, and full quantization support for large mixture-of-experts models such as DeepSeek-V3, DeepSeek-R1 and DeepSeek-R1-Zero at 671 billion parameters, including loading FP8 weights directly and running AWQ or round-to-nearest quantization on a single 80GB GPU, with export of true INT4 or INT8 quantized weights. It also supports quantization of video-generation models such as the Wan2.1 series, with export compatible with the lightx2v inference framework.
It is distributed as a Docker image, was presented at EMNLP 2024 and AAAI 2026, and recommends Python 3.11.