Build a Large Language Model (From Scratch), Chinese Edition
A Chinese translation of the book Build a Large Language Model (From Scratch), covering how GPT-style large language models are designed, trained and applied, together with its accompanying practice code.
This repository is a Chinese translation of the book Build a Large Language Model (From Scratch), an e-book that explains the principles and implementation of large language models, aimed at readers who want to understand GPT-style architectures, training and application development. It also keeps the original English e-book alongside the translated Chinese version, and includes the accompanying practice code and figures.
The book's chapters cover topics such as understanding large language models, processing text data, implementing attention mechanisms, building a GPT model from scratch for text generation, pretraining on unlabeled data, fine-tuning for classification, and instruction-following fine-tuning, with appendices on PyTorch, references, exercise solutions, advanced training-loop techniques and parameter-efficient fine-tuning with LoRA.
Translation combined an AI translation assistant, a review pass by a separate AI agent, and manual editing for accuracy.