Jason's DGX Cluster Guide
A documentation set for building and operating a two-node NVIDIA DGX Spark cluster connected by 200 Gb/s direct-attach networking, covering distributed training with NCCL, CUDA inference through EXO, NVMe-TCP shared storage, monitoring and scaling.
This guide documents a two-node NVIDIA DGX Spark (Grace Blackwell GB10) cluster connected point-to-point over a 200 Gb/s ConnectX-7 RoCEv2 link with no InfiniBand switch, reporting 93.5% DDP scaling efficiency and a 1.87x epoch speedup on distributed Llama training. It covers hardware inventory, physical cabling, OFED drivers and network configuration to get the cluster running.
Training guides cover NCCL configuration and a full data-prep-to-evaluation pipeline including LoRA merging, plus a GPU Direct RDMA guide explaining why nvidia-peermem fails and how a DMA-BUF workaround fixes it. Inference guides cover CPU inference and an OpenAI-compatible API through Exo, and working GPU-accelerated inference with Exo and tinygrad on CUDA. Storage guides cover NVMe-TCP target and initiator setup and benchmarks against NFS and local disk, and further guides cover a self-hosted ChatGPT-style deployment with Open WebUI and LiteLLM, GPU monitoring with Prometheus and Grafana, a tested model catalog, firmware update procedures, backup and recovery, and scaling from three to ten nodes.
Also includes a beginner's guide to training concepts, a standalone single-node fine-tuning path that needs no cluster, and a cost comparison between running DGX Spark hardware and renting cloud GPUs.