NVIDIA AITune
An inference toolkit for tuning and deploying deep-learning models on NVIDIA GPUs, exposing a single Python API over backends such as TensorRT, Torch-TensorRT, TorchAO, Torch Inductor and ONNX Runtime.
NVIDIA AITune compiles and converts PyTorch models and pipelines to improve inference speed and efficiency across workloads such as computer vision, natural language processing, speech recognition and generative AI. A single Python API call runs the available tuning paths, selecting the best-performing backend for the user's installed software and hardware rather than requiring the developer to choose one manually.
Supported backends include TensorRT, Torch-TensorRT, TorchAO, Torch Inductor and ONNX Runtime, and the toolkit can tune both individual models, such as ResNet or BERT, and full pipelines, such as Stable Diffusion. The resulting tuned models are intended to be deployed directly in production. The project notes that it can automatically retrieve external materials during use, which are governed by their own separate licenses that the user is responsible for reviewing.