From Tin to Tokens
A free textbook on building an LLM inference stack from scratch in Rust, covering the driver interface, attention kernels, a paged KV cache, a scheduler, multi-GPU collectives and a quantisation pipeline.
From Tin to Tokens is a textbook on inference-stack engineering, written by Dave Lemphers at Maincode, that covers the whole stack from writing ioctl structures by hand at the driver interface up to an HTTP endpoint serving thousands of concurrent users. Topics include attention kernels, a paged KV cache, a request scheduler, multi-GPU collectives, a quantisation pipeline, and the correctness apparatus used to prove that each piece works, with concepts presented ahead of the implementation details that change over time.
Each release is one numbered edition distributed as a single PDF from the project's releases page, every measurement in the book states its hardware and conditions so readers can send back specific corrections, and confirmed corrections are tracked and folded into the next edition. It is licensed under Creative Commons Attribution 4.0, and a course and reference implementation are planned to follow.