pd-bridge
A heterogeneous prefill/decode inference bridge for DeepSeek-V4-Flash that runs prefill on NVIDIA GPUs with vLLM and decode on Apple Silicon with oMLX, connected over ordinary networking.
pd-bridge connects two separate inference engines that share no cache format, framework, vendor or quantization scheme, so they can serve one large request together: prefill for the 284-billion-parameter DeepSeek-V4-Flash model runs on NVIDIA DGX Spark hardware with vLLM in FP8, while decode runs on a Mac Studio with oMLX in MXFP4. The two stages were initially linked over plain 10 gigabit Ethernet with no RDMA or Thunderbolt required.
The project reports growing the served context window over time and measures multi-hundred-thousand-token cold prompts bridging end to end in low double-digit minutes, noting that when a capture crosses the prefill side's free-memory limit it seals a valid contiguous prefix instead of failing outright. A later iteration moves the cache over RDMA across a small fabric of Mac Studios joined to the DGX Sparks' network, reducing multi-turn response latency compared with running on a single machine.
Full benchmark methodology and results are published alongside the code. It is written in Python and licensed under Apache-2.0.