Glyd
A lossless GPU-memory compression codec for AI model weights and KV caches, decoding compressed bf16 numbers on the GPU during matrix multiplication and attention so models fit in less memory.
Glyd compresses a language model's weights and KV cache in GPU memory while keeping outputs bit-for-bit identical to bf16, by holding the noisy sign and mantissa bits compactly and decoding them on the GPU inside the matrix product and attention kernels.
Benchmarks on Qwen3-32B and Qwen2.5-72B show weight sizes cut by roughly a third, fewer GPUs needed per deployment, and higher tokens-per-second throughput across a range of concurrent sequence counts, with matching accuracy on MMLU. It installs as a pip package with GPU extras and loads Hugging Face checkpoints directly, requiring a CUDA GPU of Ampere generation or later and a CUDA 12 or 13 build of PyTorch.
It is written in Rust with a Python interface, runs on Linux x86_64 and aarch64, and is licensed under BSD-3-Clause.