APEX - Adaptive Precision for EXpert Models
APEX is a mixed-precision quantization method for Mixture-of-Experts language models that assigns bit-width per tensor type and layer, producing GGUF quantizations for llama.cpp that match or beat full Q8_0 quality at a fraction of the size.
APEX classifies every tensor in a Mixture-of-Experts model by its role, routed expert, shared expert or attention, and applies a layer-wise precision gradient that gives sensitive edge layers higher precision while compressing redundant middle layers more aggressively. It produces GGUF quantizations for llama.cpp with no changes to llama.cpp's code, from an I-Quality tier at 21.3 GB down to a consumer-oriented Mini tier at 12.2 GB.
Brought by the LocalAI team, APEX is reported to beat Unsloth Dynamic 2.0 quantizations on perplexity, HellaSwag and inference speed while being about half the size. I-variants use a diverse calibration set spanning chat, code, reasoning and tool-calling rather than Wikipedia text, trading a small perplexity increase for better accuracy and lower KL divergence. Scripts are also provided to generate sensitivity and allocation configs for dense and hybrid-attention models, where the uniform MoE assumption does not directly apply.