Neural Network Compression Framework (NNCF)
A Python framework of post-training and training-time compression algorithms, such as quantization and pruning, for optimizing neural network inference in OpenVINO with minimal accuracy loss.
NNCF (Neural Network Compression Framework) provides a suite of post-training and training-time compression algorithms for optimizing the inference of neural networks in OpenVINO, aiming for minimal accuracy drop. It works with models from PyTorch, TorchFX, ONNX and OpenVINO, and is organized as a standalone Python package with a unified architecture that makes it easier to add new compression algorithms.
Post-training algorithms include quantization, weights compression and, experimentally, activation sparsity, with varying support across PyTorch, TorchFX, ONNX and OpenVINO. Training-time algorithms include quantization-aware training, weight-only quantization-aware training with LoRA and NLS, and pruning, for PyTorch. Other features include automatic, configurable model-graph transformation, a common interface across compression methods, GPU-accelerated layers for faster compressed-model fine-tuning, distributed training support, and export of compressed PyTorch models to ONNX or to SavedModel/Frozen Graph formats for use with the OpenVINO toolkit.
It is released under the Apache-2.0 license.