vLLM Docker Optimized for DGX Spark
Docker images and startup scripts for running the vLLM inference engine on NVIDIA DGX Spark hardware, from a single node to multi-node clusters over InfiniBand or RDMA.
This repository packages vLLM for NVIDIA DGX Spark systems as Docker images with startup scripts, supporting both single-node and multi-node deployments. Multi-node clusters use Ray or vLLM's native PyTorch distributed mode, with InfiniBand and RDMA (NCCL) support, direct dual-Spark connections, QSFP/RoCE switch setups and three-node mesh configurations.
It supports custom environment configuration and high-performance model loading through fastsafetensors and InstantTensor. A nightly runner image is built and tested against multiple models in both cluster and single-node configurations before being promoted, and a separate image targets larger models.
Builds can use precompiled vLLM and FlashInfer wheels, or build vLLM from a specified branch, release or commit, including private or already-checked-out repositories without recloning inside Docker.