TensorRT C++ API
A C++ library and tutorial for running GPU inference of CNN models with NVIDIA TensorRT, wrapping engine building, caching, and execution behind a small API with no exceptions.
tensorrt_cpp_api turns an ONNX model into a cached, optimized TensorRT engine and runs it through a small API that uses name-keyed tensors at its boundary, caller-owned CUDA streams, explicit host and device transfers, and a status-and-result error model instead of exceptions, with no OpenCV or TensorRT types exposed in its public headers. It targets TensorRT 10 and later, CUDA 12, and C++20 on Linux.
Its engine cache is keyed by the ONNX file's content hash together with build options, TensorRT version, and GPU identifier, so a changed model, option, driver, or GPU triggers a rebuild instead of silently reusing a stale engine. It supports dynamic input shapes through per-input optimization profiles, an engine pool for concurrent multi-stream inference, fp16, int8, and fp8 quantization, an optional fused CUDA preprocessing kernel, and optional zero-copy Python bindings for CuPy, PyTorch, or Numba GPU arrays.
It is written in C++ and released under the MIT license.