Router-weighted Expert Activation Pruning (REAP)
REAP is a research codebase for pruning experts from large sparse Mixture-of-Experts language models, using a criterion that combines router gate values and expert activation norms to compress models with less accuracy loss than merging.
REAP reproduces the expert pruning and merging methods from the paper "REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression." Its pruning criterion, Router-weighted Expert Activation Pruning, considers both router gate values and expert activation norms when deciding which experts to remove from a Sparsely-activated Mixture-of-Experts model, avoiding the functional subspace collapse the authors show expert merging introduces.
Across SMoE models from 20B to 1T parameters, REAP is reported to outperform merging and other pruning methods on generative benchmarks, especially at 50% compression, and to achieve near-lossless compression on code-generation and tool-calling tasks with Qwen3-Coder-480B and Kimi-K2. The repository includes a memory-efficient, layer-wise calibration observer for pruning large models on a single GPU, and a released data-calibration mix for agentic reasoning models.