# Building GLM 5.2 in C to Run a 744B Model on Consumer Hardware

A from-scratch pure C inference engine that runs GLM-5.2, a 744-billion-parameter mixture-of-experts model, on a single machine with about 16 GB of RAM by quantizing to int4 and streaming inactive experts from disk.

This repository documents and implements an inference engine for GLM-5.2, a 744-billion-parameter mixture-of-experts language model that is roughly 1.5 terabytes in bfloat16. Because only 8 of its 256 experts per layer fire for any given token, the engine keeps the always-active part of the model in memory and streams the sleeping experts from disk, so the working set stays small enough to run on a laptop with about 16 GB of RAM and no GPU.

The write-up and accompanying code cover quantizing the vendor's FP8 weights to int4, validating a smaller streaming engine token for token against PyTorch, implementing the forward pass with multi-head latent attention and a Mixture-of-Experts router, writing integer dot-product kernels checked against a plain C reference, building a multi-tier memory system across VRAM, RAM, cache and disk, and speculative decoding.

The engine, conversion tools and benchmark harness are derived from the colibri project and released under the Apache-2.0 license.

Website: https://medium.com/@fareedkhandev/77a3df56e7b4