Qwen3.8-Flash-Next-NVFP4-vLLM-Dual-DGX-Spark
This project provides scripts and documentation for running multi-node inference of Qwen3.8-Flash-Next in NVFP4 across two NVIDIA DGX Spark nodes using vLLM with tensor parallelism, expert parallelism and multi-token prediction.
This repository documents and scripts multi-node inference of the Qwen3.8-Flash-Next model, in NVFP4 precision, across two NVIDIA DGX Spark nodes connected over ConnectX RoCE or InfiniBand, using vLLM configured with tensor parallelism, expert parallelism and three-way multi-token prediction. It builds on an earlier single-node DGX Spark recipe for the same model family.
A start script launches a named container on both nodes, distributing model weights either by syncing a copy from the head node over rsync or, optionally, by sharing them over NFS to skip the worker copy entirely; flags allow skipping the download or the launch step independently. An alternate lane runs the same two-node setup against a stock upstream vLLM release. The documentation is explicit that its reported KV-cache budget and runtime numbers are measurements read from the running server's own logs, not estimates, and it requires roughly 126 GiB of free space per node for the checkpoint.