TensorRT
The open-source components of NVIDIA's TensorRT SDK for high-performance deep-learning inference, including plugins, an ONNX parser, sample applications and import-workflow guides.
This repository holds the open-source components of NVIDIA TensorRT, an SDK for high-performance deep-learning inference on NVIDIA GPUs: TensorRT plugins, an ONNX parser, and sample applications demonstrating the platform, as a subset of the full TensorRT general-availability release plus some extensions and fixes.
TensorRT 11 streamlines the API compared with the previous major version: weakly typed networks are replaced by strongly typed networks, implicit quantization is replaced by explicit quantization, the older plugin interface is replaced by a newer one, the TREX profiling tool is replaced by Nsight Deep Learning Designer, and support for Python 3.9 and older is dropped. Guides cover import paths from ONNX, Torch-TensorRT, Hugging Face and Optimum, and the Network Definition API, along with a support matrix across large language, encoder, vision, audio, diffusion and multimodal models.
The project also ships agentic coding skills for TensorRT usage and benchmarking, for use with AI coding agents.