Model Compression Toolkit
An open-source toolkit for quantizing and compressing PyTorch and Keras neural networks for efficient, constrained hardware, with post-training quantization, gradient-based fine-tuning and quantization-aware training.
Model Compression Toolkit (MCT) provides quantization and compression tools for deploying neural networks on constrained hardware. It supports post-training quantization (PTQ), gradient-based parameter fine-tuning (GPTQ) and quantization-aware training (QAT), each with different accuracy, complexity and compute trade-offs, ranging from a few CPU minutes to tens of GPU hours.
Its quantization core includes mixed-precision search that assigns an optimal bit-width per layer, graph optimizations for the quantization process, and quantization parameter search using methods such as MSE, no-clipping and MAE to minimize noise. It takes a pre-trained PyTorch or Keras model as input, with an optional representative dataset or built-in data generation, and installs with pip on Python 3.10 or newer.