Modern LLM Notebook
A notebook-first course of 26 runnable Jupyter notebooks that builds a modern LLM system from scratch in PyTorch, covering tokenizers, attention, mixture-of-experts, training, RLHF, quantization, speculative decoding, and deployment.
Modern LLM Notebook has learners implement the essential machinery of a large language model themselves rather than treating it as a black box: tokenizer, embedding, self-attention, Transformer blocks, training objectives, mixture-of-experts layers, LoRA, RLHF, decoding, KV caching, and long-context techniques, following the complete path from raw text to a working model system across 26 notebooks.
Its recently rebuilt inference section covers FP8 and FP4 quantization formats with a grid experiment and an end-to-end walkthrough producing several standard quantized checkpoint formats and serving each one; a runnable speculative-decoding loop with measured acceptance rate and speedup; batching, paging, and prefix-caching simulators alongside refreshed serving-engine deployment workflows; and an evaluation notebook that scores real models against common benchmarks and registers a custom benchmark into an existing evaluation harness.
Each notebook keeps a consistent learning loop of intuition, hand calculation, implementation, and experiment, and the course is bilingual, with Chinese as the source edition and an English mirror updated alongside it; it can be run locally or started directly in a hosted notebook environment.