LLMs-Zero-to-Hero
A Chinese-language tutorial series that teaches how to build large language models from scratch, covering pretraining, fine-tuning, mixture-of-experts, RLHF and deployment, with accompanying notebooks and videos.
LLMs-Zero-to-Hero is a self-study course on building large language models from scratch, written in Chinese with accompanying Jupyter notebooks and video lectures.
It works through the fundamentals of LLM training in stages: pretraining, supervised fine-tuning, direct preference optimization and reinforcement learning from human feedback, plus building a code-focused LLM, mixture-of-experts models, and deployment topics such as inference optimization and quantization. Chapters pair a written explanation with a runnable notebook that reproduces the technique from scratch.
The material is organized into per-chapter folders, each with its own README and code, and continues to be updated as new chapters are written toward a planned book. It is released under the Apache-2.0 license.