Gemma 3 MoE builder
A toolkit that converts a dense Gemma-3-270m model into a mixture-of-experts model, then trains, inspects and serves it on CPU through llama.cpp.
gemma3moe-builder is a small toolkit that sparse-upcycles a dense Gemma-3-270m checkpoint into a routed mixture-of-experts model, made of a linear router and several expert networks selected top-k with renormalized weights, then serves it on CPU through llama.cpp. Mixture-of-experts adds parameter capacity while keeping the active compute, and therefore CPU speed, close to the original dense model.
It supports diverse expert initialization by seeding experts from different Gemma-3-270m variants, continued training or fine-tuning with a plain Hugging Face Trainer with optional load-balance and z-loss terms, and inspection tools for routing balance, dead experts and per-task specialization. A trimmed 8-expert build runs at roughly 90 to 210 tokens per second on Apple Silicon and quantizes down to about 460 MB. It installs with pip.