# ds4-on-spark

ds4-on-spark packages a Blackwell-CUDA performance fork of the ds4 engine to serve DeepSeek-V4-Flash entirely on-device on an NVIDIA DGX Spark, with large prefill and decode speedups over the upstream engine.

ds4-on-spark installs Entrpi/ds4, a major-feature fork of antirez/ds4 (DwarfStar 4), which builds the upstream inference engine into a batched, multi-request server tuned for Blackwell CUDA hardware, targeting DeepSeek-V4-Flash served entirely on-device on an NVIDIA DGX Spark, with RTX PRO 6000 and 5090-class GPUs also supported. One command installs and starts serving the model; upstream remains the architectural foundation and model recipe, and this project pins and packages the fork.

Compared with the upstream engine, the fork reports two to three times the prefill throughput and roughly one and a third times the decode speed across a wide context range, plus continuous batching, disk-persisted prefix caching that survives restarts, and lossless speculative decoding. A memory governor tracks the engine's actual memory use so requests are charged what they will use, idle memory returns to the pool automatically, and an out-of-memory condition produces a clear refusal instead of a crash.

Website: https://github.com/Entrpi/ds4-on-spark