Whallm
A macOS app that runs large language models such as DeepSeek V4 and Qwen3.8 on Apple Silicon by reading model experts from SSD, with a built-in chat interface and an OpenAI-compatible API.
Whallm runs large mixture-of-experts language models on Apple Silicon Macs by streaming the experts a request needs from SSD instead of holding the entire model in memory. It supports DeepSeek V4, DeepSeek V4.1 and Qwen3.8, and exposes both a built-in chat interface and an OpenAI-compatible API server.
Acceleration techniques vary by model: DeepSeek V4 uses layer-by-layer input processing, batched expert calculations, an FP8 key-value cache, and optional Apple Neural Engine projection; DeepSeek V4.1 adds packed key-value and index caches and candidate-only index scoring. The server keeps common weights in memory and loads model-specific rows from disk only as needed. Published benchmarks on M5 Pro and M2 Max Macs show prefill and decode throughput along with peak memory use for each supported model and chipset.
The app requires Apple Silicon with macOS 15 or later and 64 GiB of unified memory recommended, and defaults to serving on a local port; model weights are downloaded separately and are not bundled with the app.