Transformers & LLMs cheatsheet for Stanford's CME 295
A VIP cheatsheet summarizing Stanford's CME 295 course on Transformers and Large Language Models, covering self-attention, fine-tuning methods, distributed training and applications such as AI agents.
This VIP cheatsheet summarizes the material covered in Stanford's CME 295 Transformers and Large Language Models course. It covers transformer architecture and variants, self-attention, and LLM techniques such as prompting, fine-tuning methods including SFT and LoRA, and preference-tuning methods including RLHF and DPO.
It also covers optimizations such as distributed training, KV caching and speculative decoding, and applications including AI agents, evaluation and extensions such as diffusion-based LLMs. The cheatsheet is a condensed companion to a longer book, "Super Study Guide: Transformers & Large Language Models," and is translated into more than a dozen languages.