Optimum
A Hugging Face library that optimizes and accelerates Transformers, Diffusers, TIMM and Sentence Transformers models for training and inference on specific hardware.
Optimum extends Hugging Face's Transformers, Diffusers, TIMM and Sentence Transformers libraries with tools that optimize models for efficiency on targeted hardware, while keeping the same easy-to-use interfaces.
It provides accelerated inference paths through ONNX and ONNX Runtime, OpenVINO for Intel hardware, and ExecuTorch for on-device inference on mobile and edge devices, alongside dedicated support for accelerators such as Intel Gaudi, AMD Instinct GPUs and Ryzen AI NPUs, NVIDIA TensorRT-LLM, AWS Trainium and Inferentia, and FuriosaAI. Each accelerator has its own installable extra, such as optimum[onnxruntime] or optimum[habana], selected when installing the package.
It is installed with pip and released under the Apache 2.0 license.