pi-quant: Prime Intellect Fast Quantization Library
A fast, multithreaded CPU library of SIMD-optimized quantization and dequantization kernels for machine learning, more than twice as fast as PyTorch's built-in routines on tested hardware.
pi-quant's kernels use architecture-specific SIMD intrinsics -- SSE4.2, AVX2 and AVX512F on AMD64, and Neon on ARM64 -- with the fastest available kernel chosen automatically at runtime through CPU feature detection. They convert 32- and 64-bit floats to and from 4-, 8-, 16-, 32- and 64-bit integers with a choice of nearest or stochastic rounding, and support both set and add store modes during dequantization, which is useful for ring-reduction operations.
A reusable thread pool keeps multithreading overhead low, and a C99 API makes the library usable from C projects or other languages through foreign-function bindings, alongside a Python API that works directly with PyTorch or NumPy. It installs from PyPI as pypiquant, and its own benchmarks measure it against PyTorch's built-in per-tensor quantization routines on several CPU architectures.