Happy-LLM
A systematic, free tutorial that teaches how large language models work by building one from scratch, from NLP and Transformer fundamentals through pretraining, fine-tuning, RAG, agents and reinforcement learning.
Happy-LLM is organized as eight chapters plus supporting material: an introduction to NLP basics, the Transformer architecture and attention mechanism built by hand, a comparison of encoder-only, encoder-decoder and decoder-only pretrained language models, and the general characteristics and training strategy of large language models. Later chapters have the reader implement a LLaMA2 model and its tokenizer and pretrain a small model, then cover the full training pipeline of pretraining, supervised fine-tuning and parameter-efficient methods such as LoRA and QLoRA, followed by applications including model evaluation, retrieval-augmented generation and agents, and a chapter on agentic reinforcement learning methods including GRPO, OPD and Search-R1.
It is aimed at university students, researchers and other LLM learners with some Python programming background, ideally with prior exposure to deep learning and NLP concepts. The book is distributed free, with an online reading version, a downloadable PDF, companion slides, and small pretrained and fine-tuned example models.
It is delivered mainly as Jupyter notebooks.