DFlash vLLM for DGX Spark
A plug-and-play Docker image that runs vLLM with DFlash block-diffusion speculative decoding on NVIDIA Blackwell hardware such as the DGX Spark, optionally combined with TurboQuant KV-cache compression, to speed up decoding on Blackwell-exclusive NVFP4 tensor cores.
This project packages block-diffusion speculative decoding, called DFlash, into a ready-to-run vLLM Docker image for NVIDIA Blackwell GPUs, including the DGX Spark (GB10), B200, GB200 and RTX 5090; it will not run on Hopper, Ampere or earlier NVIDIA architectures because NVFP4 tensor cores are Blackwell-exclusive. TurboQuant KV-cache compression can optionally be stacked on top, though it is not included in the default image.
Its documentation is explicit about operational caveats: first boot after a model is on disk takes five to ten minutes for weight loading, CUDA graph capture and NVFP4 autotuning, with subsequent boots faster from cached artifacts; the first inference request takes about thirty seconds even after the server reports healthy, because vLLM performs one-time CUDA graph specialization on that request, so a few warm-up requests are recommended before measuring latency; and DFlash's speedup is content-dependent, faster on code and reasoning than on free prose, with no speedup on adversarial random-token input.
It requires an aarch64 Blackwell GPU with at least 64 GB of free unified or GPU memory, at least 50 GB of free disk, NVIDIA driver 580.x or later, and Docker 25 or later with the NVIDIA container toolkit; it is licensed under Apache-2.0.