LLM-RL-Visualized
A collection of more than 100 original diagrams explaining large-language-model and reinforcement-learning algorithms, from model architecture through fine-tuning methods to policy-optimization algorithms.
These diagrams cover an overall technical landscape of large models and reinforcement learning, LLM and VLM structure, supervised fine-tuning topics such as LoRA, prefix-tuning, token-ID mapping and instruction-data packing, and training-free optimization techniques such as chain-of-thought prompting, beam search and retrieval-augmented generation.
Further diagrams cover direct preference optimization compared with RLHF, reinforcement-learning fundamentals such as Markov decision processes, exploration-exploitation, Monte Carlo and temporal-difference methods, and DQN, and policy-optimization architectures and their derivatives, including actor-critic methods, generalized advantage estimation, PPO, TRPO, PPO-Clip, deterministic policy gradients and DDPG, continuing into RLHF and RLAIF.
Diagrams are provided as high-resolution images and as scalable SVG vector files; a companion book covering the same material in more depth is sold separately. It is written in Python.