vLLM on DGX Spark Cluster
Deployment scripts for running vLLM across one to many NVIDIA DGX Spark systems, connected directly by cable or through a switched fabric, with tensor parallelism that scales automatically with the cluster size and more than 40 tested model presets.
This project deploys vLLM across one to many NVIDIA DGX Spark systems for serving or benchmarking large language models: a single Spark, two Sparks connected by a direct QSFP cable, or three or more Sparks over a switched fabric, with tensor parallelism scaling automatically to match the number of workers. It is an independent community effort, not affiliated with or supported by NVIDIA or the vLLM project.
A single command starts the entire cluster from the head Spark over SSH, auto-detecting InfiniBand IPs, network interfaces and HCA devices for multi-Spark setups, with InfiniBand RDMA used for inter-Spark communication. It ships 41 model presets, 16 community-tested and 25 drawn from NVIDIA's official Spark vLLM support matrix, covering FP8, NVFP4, MXFP4 and BF16 quantizations, along with a benchmarking mode with multiple test profiles and an offline mode.
It runs inside an NVIDIA container image bundling vLLM, PyTorch, CUDA and Ray, with the cluster scripts installing a matching Ray version into the head and worker containers at startup; single and two-Spark configurations are verified end to end, while paths for more than two Sparks are reviewed but not yet exercised on real hardware.