vLLM-Omni
An inference framework built on vLLM that extends serving to omni-modality models covering text, image, audio, and video generation and understanding.
vLLM-Omni adds serving support for models that go beyond text-only autoregressive generation, including diffusion, image, audio, and video generation, and joint audio/video output. It rebases regularly on upstream vLLM stable releases and adds engine-owned sessions for full-duplex serving, cross-stage KV and multimodal payload transfer between stages, and connectors for distributed inference backends such as Mooncake.
The project tracks a wide range of omni-modality and world models, including MiniCPM-o, MiniMax, LingBot World, and Qwen3-Omni, and supports request-level and step-wise batching for diffusion reinforcement learning workloads through VeRL-Omni. It runs on NVIDIA, AMD, and other accelerators depending on the release.
It is written in Python and distributed under the Apache-2.0 license.