cider
An MLX extension for Apple Silicon that adds INT8/INT4 activation quantization and an optimized decode attention kernel to speed up LLM prefill and autoregressive decode on M5 Macs.
cider builds on MLX to add online activation quantization (W8A8/W4A8) that MLX itself does not provide, through custom INT8 TensorOps kernels implemented as MLX custom primitives with full lazy evaluation, and an optimized scaled dot-product attention decode kernel inspired by FlashInfer that uses contiguous chunk GQA scheduling, register tiling and, for long sequences, a two-pass FlashDecoding reduction.
Its INT8 C++ extension is conditionally compiled: it only builds on Apple M5 and newer, where pip install -e . performs a full CMake and Metal kernel build; on M4 and earlier it installs a pure-Python package where the quantization features report unavailable. It also patches mlx_vlm (validated on version 0.4.3), including fixes for Qwen3-VL multi-image inference around RoPE position handling and chunked prefill.