OpenVINO Model Server
A production-grade C++ inference server, built on OpenVINO and optimized for Intel hardware, that serves generative AI, agentic and classic deep-learning models over OpenAI-compatible and KServe network APIs.
OpenVINO Model Server (OVMS) serves both generative AI and agentic workloads, large language models, vision-language models, image generation and audio, as well as classic deep-learning models such as object detection, classification and OCR, exposing them over standard network APIs. It offers an OpenAI-compatible API for text generation, embeddings, image generation and audio, and KServe APIs for classic model inference, and runs on Docker, bare metal, Kubernetes or OpenShift, and Windows.
It is Intel-optimized, accelerating inference on CPU, GPU and NPU through OpenVINO. Serving an LLM is a single Docker or Windows binary command that downloads a model automatically from Hugging Face into a models folder and exposes an OpenAI-compatible REST endpoint on a chosen port; serving a classic model with the KServe API works the same way, pointing at a downloaded model file with layout and normalization parameters.
GPU acceleration on Linux requires passing through the Intel GPU render device to the container.