Qwen3.8-27B 4-bit on a single DGX Spark (GB10 / SM121)
A measured serving recipe and benchmark comparing two 4-bit quantized checkpoints of a 27-billion-parameter Qwen model against an FP8 baseline on a single NVIDIA DGX Spark, showing where quantization helps and where it stops mattering.
This repository is a benchmark of serving a 4-bit quantized Qwen3.8-27B language model on a single DGX Spark (GB10) system, comparing two different 4-bit quantization approaches against an FP8 baseline, with every reported number generated from a benchmark harness included in the repository. It measures speed only; no answer-quality evaluation was performed, and any accuracy figures mentioned are cited from other publishers and left unverified.
The headline results are 75 tokens per second in single-stream decoding and 246 tokens per second in aggregate at eight-way concurrency, using two different draft-depth configurations that trade one result against the other. A key finding is that the throughput advantage from quantization shrinks as concurrency rises, from roughly 27 percent faster at single-stream to about 0.2 percent faster at sixteen-way concurrency, because decoding shifts from being limited by memory bandwidth to being limited by compute as more sequences are batched together, and 4-bit quantization also costs some prefill throughput at long prompt lengths.