Intel Neural Compressor
An open-source library from Intel for quantizing and compressing deep-learning and large language models across PyTorch, TensorFlow, and ONNX Runtime.
Intel Neural Compressor is an open-source Python library supporting popular model-compression techniques on mainstream deep-learning frameworks, including PyTorch, TensorFlow, and JAX. It provides static and dynamic quantization, SmoothQuant, weight-only quantization, quantization-aware training, mixed precision, pruning, sparsity, auto-tuning, and knowledge distillation.
Through integration with a related project, it supports advanced low-bit quantization, including INT8, FP8, INT4, and newer MXFP8/MXFP4/NVFP4 formats, for large language and vision-language models such as LLaMA, Qwen, DeepSeek, and Flux. It targets a wide range of Intel hardware, including Gaudi AI accelerators, Core Ultra and Xeon processors, and Data Center GPUs, with extensive testing, plus limited testing on AMD and ARM CPUs and NVIDIA GPUs.
It is released under the Apache License 2.0, with separate PyTorch and TensorFlow backend packages published on PyPI.