cpubrrr
A research runtime, written in Rust, that runs mixture-of-experts language models entirely on an Apple Silicon CPU, with hand-written NEON and SME kernels and no GPU code path, benchmarked against llama.cpp with corrected, re-measured comparisons.
cpubrrr is a from-scratch CPU-only inference runtime for mixture-of-experts models on Apple M4 Max hardware. Its binary links only the C standard library, so it cannot use the GPU, and it decodes four mixture-of-experts models across two architectures and two quantization formats from one configuration-driven engine, starting from an initial gpt-oss:20b experiment.
The project publishes detailed, same-session comparisons against upstream llama.cpp, including a documented correction where an earlier claim of parity was based on a mistuned upstream baseline; measured against a properly tuned upstream at its best thread count, cpubrrr's current engine decodes gpt-oss:20b about 1.17 times faster, with accuracy unchanged on a GSM8K check against the same weights served through Ollama.
It is written in Rust and licensed under Apache-2.0.