SGLang-Omni
A serving framework for audio and multimodal AI models that runs text-to-speech, speech recognition, and omni chat models behind an OpenAI-compatible API, built to handle multi-stage generation pipelines.
SGLang-Omni is a multi-stage serving runtime for omni, speech, and text-to-speech models. It models generation as a pipeline of stages, such as preprocessing, encoders, autoregressive engines, talkers, decoders, and vocoders, each running behind a scheduler suited to its workload.
A control plane coordinates requests while a separate data plane moves tensor payloads between stages over shared memory, NCCL, NIXL, or Mooncake backends. It composes with SGLang for autoregressive scheduling and model execution where applicable.
It exposes OpenAI-compatible endpoints for multimodal chat, speech generation, batch and streaming speech, uploaded voices, and transcription, serving models such as Qwen3-Omni, Ming-Omni, and MiniMax Music 3. It is written in Python and distributed under the Apache-2.0 license.