ncnn
A high-performance neural network inference framework optimized for mobile, embedded and desktop deployment, with no third-party runtime dependencies and both CPU and Vulkan GPU backends.
ncnn is a neural network inference framework optimized for mobile, embedded and desktop deployment. It has no third-party runtime dependencies, runs across CPU and Vulkan GPU backends, and includes a tool, pnnx, for converting PyTorch and ONNX models to its own format so they can be deployed on phones, PCs, browsers and edge devices.
The recommended path for a new model is PyTorch, converted through pnnx, into ncnn's own param and bin files, which can then be loaded and run through either a C++ API or a Python API. It is used in several of Tencent's own applications, including QQ, Qzone and WeChat.
It is written in C++, builds for platforms including Linux, Windows, macOS, Android, iOS, HarmonyOS and WebAssembly, and is distributed on GitHub Releases.