EOQ: Entropy-Optimal Quantization for LLMs
A quantization method for large language models that rotates weights toward a Gaussian distribution before applying optimal centroids, shrinking a model's memory footprint with close to full-precision accuracy.
EOQ's core technique, called PolarQuant, quantizes a model's weights in three steps: block-normalizing weights by their L2 norm, applying a Walsh-Hadamard rotation that reshapes the weight distribution to approximate a Gaussian, and then quantizing with Lloyd-Max centroids, which are optimal for that distribution. Compact storage keeps one byte per weight for the resulting integer codes and norms.
On a Qwen3.5-9B model, the project's own benchmarks report the Hadamard rotation step as responsible for about 98 percent of the accuracy improvement over plain quantization, with Lloyd-Max centroids adding a smaller additional gain on top of rotation; a recommended configuration combining the method with 4-bit integer quantization is reported to use roughly a third of the VRAM of an FP16 baseline at a small accuracy cost. The method also runs on Apple Silicon through MLX with 4-bit quantization.
It is written in Python, and an accompanying arXiv preprint describes the method.