dmlx
A native C and Metal inference engine that runs large mixture-of-experts language models, such as a 284-billion-parameter DeepSeek model, on a single Apple Silicon Mac by streaming quantized experts from disk instead of holding everything in memory.
dmlx runs frontier mixture-of-experts language models on Apple Silicon through a native C and Metal inference path with packed expert weight streaming, rather than a Python-based stack such as mlx-lm. Its stated design goal is to run models that would otherwise run out of memory on ordinary hardware, by packing quantized experts on SSD and keeping only a bounded cache of them resident.
A published benchmark reports a 284-billion-parameter DeepSeek model running at just over one token per second on a Mac with 48 gigabytes of memory, with detailed timing, latency and correctness figures recorded in a machine-readable artifact alongside a human-readable report. Compared to a Python-based alternative, the project reports a much smaller binary size, several switchable key-value cache strategies, larger effective context through combined RAM and SSD use, and the ability to embed the engine in an iOS app through its C ABI.