Understanding AI Infra: Quantitative Analysis and System Design
An open-source book that derives LLM inference and training system design from hardware constraints and model architecture, with quantitative worked estimates and companion calculation tools.
This book is a sister volume to an earlier, widely starred book on AI agent design by the same author. It argues that understanding AI infrastructure now requires the same quantitative habit that computer architecture teaches: deriving a system's design from constraints such as compute, memory, communication and dependencies, checked against hardware capacity, bandwidth and throughput.
Its twelve chapters cover an introduction to AI infrastructure, model architecture, inference and training workloads, accelerator architecture, operators and runtimes, superclusters, datacenter networking, inference optimization, distributed inference, training systems, resource scheduling, and edge-cloud collaboration.
Companion calculation tools run on the Python 3.10+ standard library with no GPU or model weights required, invoked as python3 calculations/calc.py models or forward; the book is available as PDF, EPUB and online reading in Simplified Chinese, English and Traditional Chinese, and is licensed under Apache-2.0.