# vLLM Docker Optimized for DGX Spark

Docker images and startup scripts for running the vLLM inference engine on NVIDIA DGX Spark hardware, from a single node to multi-node clusters over InfiniBand or RDMA.

This repository packages vLLM for NVIDIA DGX Spark systems as Docker images with startup scripts, supporting both single-node and multi-node deployments. Multi-node clusters use Ray or vLLM's native PyTorch distributed mode, with InfiniBand and RDMA (NCCL) support, direct dual-Spark connections, QSFP/RoCE switch setups and three-node mesh configurations.

It supports custom environment configuration and high-performance model loading through fastsafetensors and InstantTensor. A nightly runner image is built and tested against multiple models in both cluster and single-node configurations before being promoted, and a separate image targets larger models.

Builds can use precompiled vLLM and FlashInfer wheels, or build vLLM from a specified branch, release or commit, including private or already-checked-out repositories without recloning inside Docker.

Website: https://github.com/eugr/spark-vllm-docker