slotstream
A native Swift engine that streams a 125-billion-parameter mixture-of-experts model from SSD and caches its busiest experts in memory, so it can run on Apple Silicon Macs that cannot hold the whole model in RAM.
Slotstream runs a 125-billion-parameter open mixture-of-experts model on Macs with 16 to 64 GB of memory. It keeps most of the model on SSD and loads the parts it needs as it writes, reporting about 16 tokens per second on a 48 GB M5 Pro at a 22 GB memory target.
Users can chat with it, ask about images, or use it as a coding model: a launch command starts Claude Code against the local model, and other agent harnesses work the same way. Developers can also connect through Ollama-, OpenAI-, and Anthropic-compatible APIs, or a Swift library.
The whole engine is a single native Swift program built on Apple's MLX and Metal, and works offline with no Python and no cloud account after the initial download. It requires an Apple Silicon Mac with at least 16 GB of memory, macOS 14 or later, and about 110 GB of free SSD space; Windows, Linux, and Intel Macs are not supported.