Olive
An AI model optimization toolkit for the ONNX Runtime that, given a model and target hardware, composes fine-tuning, conversion, quantization and other optimization techniques to produce the most efficient ONNX model for inference on CPUs, GPUs and NPUs.
Olive (short for ONNX LIVE) is a toolkit for optimizing machine learning models for the ONNX Runtime. Given a model and a target piece of hardware, it composes the optimization techniques best suited to that combination, such as fine-tuning, format conversion and quantization, to produce an efficient ONNX model for cloud or edge inference while respecting constraints such as accuracy and latency.
It installs as a Python package with a command-line interface and supports quantization algorithms for CPUs, GPUs and NPUs, including recent additions for small language models and MultiLoRA serving. A VS Code extension, AI Toolkit, also uses Olive to fine-tune models from within the editor.