# vLLM-Omni

An inference framework built on vLLM that extends serving to omni-modality models covering text, image, audio, and video generation and understanding.

vLLM-Omni adds serving support for models that go beyond text-only autoregressive generation, including diffusion, image, audio, and video generation, and joint audio/video output. It rebases regularly on upstream vLLM stable releases and adds engine-owned sessions for full-duplex serving, cross-stage KV and multimodal payload transfer between stages, and connectors for distributed inference backends such as Mooncake.

The project tracks a wide range of omni-modality and world models, including MiniCPM-o, MiniMax, LingBot World, and Qwen3-Omni, and supports request-level and step-wise batching for diffusion reinforcement learning workloads through VeRL-Omni. It runs on NVIDIA, AMD, and other accelerators depending on the release.

It is written in Python and distributed under the Apache-2.0 license.

Website: https://docs.vllm.ai/projects/vllm-omni