# Qwen3.8-27B on RTX 5090

A reproducible serving configuration, vLLM patches, and measured benchmarks for running the Qwen3.8-27B model on one or two RTX 5090 GPUs at 262K context for concurrent coding agents.

This repository documents a specific serving configuration for running the Qwen3.8-27B model on one or two RTX 5090 GPUs with a 262K token context window, targeting a workload of several concurrent coding agents with long contexts alongside interactive chat with vision, reasoning, tool calling, and structured output. It uses NVFP4-quantized weights, vLLM with a pinned NVFP4 KV cache, the checkpoint's own multi-token-prediction draft head, and patches for cross-GPU all-reduce and batch-sharded sampling.

Every measurement is stated to be taken on one machine on a given date, with the raw results directory named alongside each figure rather than estimated. Reported figures include aggregate decode throughput up to about 4,497 tokens per second of code at 64 concurrent streams, and cold prefill throughput starting around 8,365 tokens per second that falls as prompt length grows because full-attention layers reread the whole prefix per chunk.

Launch scripts and a plotting script generate the numbers from raw records checked into the repository.

Website: https://github.com/adrienbrault/qwen3.8-27b-rtx5090