# Godwit

A Swift and Metal runtime that runs a 120-billion-parameter mixture-of-experts language model on a 16 GB Apple Silicon Mac by streaming only the experts a token actually needs from disk.

Godwit keeps only a mixture-of-experts model's shared trunk, roughly 2 gigabytes for GPT-OSS-120B, resident in memory, and streams the individual expert weights a token's router selects from SSD on demand, rather than requiring the full, much larger model to be loaded, which is what most inference runtimes still assume. The tradeoff is throughput: on a base MacBook Air the approach currently runs at roughly 1.4 tokens per second, in exchange for making a 59-gigabyte model runnable at all on a 16-gigabyte machine.

It also runs GPT-OSS-20B and Qwen3-30B-A3B from the same binary, switchable at runtime, and serves a live view of which of a model's thousands of experts are firing for each token, built with Swift and Metal and no external dependencies. Its documentation corrects an earlier, incorrect conclusion about expert-choice prefetching, explaining why naively reusing a previous layer's expert choices does not work but running the next layer's router early on the current hidden state can.

Website: https://rayl15.github.io/Godwit/