OpenVINO
An open-source toolkit for optimizing and deploying deep-learning models across CPUs, GPUs and AI accelerators, supporting models from PyTorch, TensorFlow, ONNX and other frameworks.
OpenVINO is an open-source toolkit for optimizing and deploying deep-learning models. It speeds up inference for computer vision, automatic speech recognition, generative AI, natural-language processing and other common tasks using large and small language models.
It accepts models trained with PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle and JAX/Flax, and can pull models built with the Hugging Face transformers and diffusers libraries directly through Optimum Intel, converting and deploying them without their original frameworks. It runs on x86 and ARM CPUs, Intel integrated and discrete GPUs, and Intel NPU accelerators, from edge devices to the cloud.
It installs as a Python package, provides APIs in C++, Python, C and Node.js, and includes a GenAI API for optimized model pipelines. It is written in C++ and licensed under Apache-2.0.