Nari Qwen3-TTS
A high-performance serving implementation of Qwen3-TTS that streams speech over HTTP and WebSocket with under 50 milliseconds to first audio at 10 requests per second on a single H100.
Nari Qwen3-TTS is a serving implementation of the Qwen3-TTS 1.7B CustomVoice text-to-speech model tuned for latency, exposing both streaming and non-streaming speech generation over HTTP, plus WebSocket-based input streaming so text can be fed in incrementally. On a single NVIDIA H100 SXM it reports 10 requests per second with sub-50ms p95 time-to-first-audio while maintaining real-time playback, and sub-80ms p95 time-to-first-audio even at 20 requests per second.
It runs as a Docker container requiring an NVIDIA H100, the NVIDIA Container Toolkit, and a compatible CUDA driver, or can be run directly with a pinned uv/Python environment after installing a few system packages. Three serving profiles are provided: one that prioritizes latency for the first audio chunk, a balanced default that trades off first-audio latency against sustained throughput, and one that favors larger batches for throughput. The engine has been tested only with English as the primary language, and the project publishes a benchmark repository and blog post describing its methodology against other serving engines.