# Glyd

A lossless GPU-memory compression codec for AI model weights and KV caches, decoding compressed bf16 numbers on the GPU during matrix multiplication and attention so models fit in less memory.

Glyd compresses a language model's weights and KV cache in GPU memory while keeping outputs bit-for-bit identical to bf16, by holding the noisy sign and mantissa bits compactly and decoding them on the GPU inside the matrix product and attention kernels.

Benchmarks on Qwen3-32B and Qwen2.5-72B show weight sizes cut by roughly a third, fewer GPUs needed per deployment, and higher tokens-per-second throughput across a range of concurrent sequence counts, with matching accuracy on MMLU. It installs as a pip package with GPU extras and loads Hugging Face checkpoints directly, requiring a CUDA GPU of Ampere generation or later and a CUDA 12 or 13 build of PyTorch.

It is written in Rust with a Python interface, runs on Linux x86_64 and aarch64, and is licensed under BSD-3-Clause.

Website: https://github.com/surya-koritala/Glyd