# Swiftlet

Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts language models, including 35B and 80B parameter variants, on ordinary Apple devices such as iPhones, by streaming routed expert weights from storage instead of holding the whole model in memory.

Swiftlet runs the Qwen3-Next and Qwen3.5/3.6 Mixture-of-Experts hybrid model family locally on Apple hardware by keeping only the small dense core of the model resident in memory and streaming the routed expert weights from disk on demand. This lets models as large as 35B and 80B parameters run with peak memory in the single-digit gigabytes, including on devices that could not otherwise hold the full model.

Reported figures on an Apple M5 show the 4-bit 35B model decoding at 7 to 11 tokens per second in 2.6GB of peak RAM and 18GB of disk, and the 4-bit 80B model decoding at 4.5 to 5 tokens per second in 4.3GB of peak RAM and 42GB of disk; a base M1 with 16GB of RAM decodes the 4-bit 35B model at about 2.45 tokens per second.

It is written in Swift and licensed under Apache-2.0.

Website: https://github.com/leonickson1/Swiftlet