# spark_vllm_docker

A unified vLLM serving configuration, distributed as a Docker Compose stack, for running large language models on NVIDIA DGX Spark GB10 hardware, supporting either a single-node setup or two nodes joined over a 200 Gbps RDMA link.

spark_vllm_docker packages a vLLM serving stack for NVIDIA DGX Spark (GB10) systems from one repository, Dockerfile and Compose file, switched between a single-Spark topology needing no RDMA setup and a dual-Spark topology using a 200 Gbps RoCE or InfiniBand link with either a Ray or a plain multiprocessing backend, selected with a CLUSTER_MODE environment variable.

The repository documents several validated model-serving configurations, such as DeepSeek-V4-Flash and Qwen3.8-Flash-Next, each recorded with its status, backend and an immutable container image digest; because only one model is actually served on a given deployment at a time, the documentation is explicit that its own index describes configuration options rather than which model is currently running, and that live state should be checked against the running server instead.

It is a Python and Docker project licensed under Apache-2.0.

Website: https://github.com/bjk110/spark_vllm_docker