# LLM-RL-Visualized

A collection of more than 100 original diagrams explaining large-language-model and reinforcement-learning algorithms, from model architecture through fine-tuning methods to policy-optimization algorithms.

These diagrams cover an overall technical landscape of large models and reinforcement learning, LLM and VLM structure, supervised fine-tuning topics such as LoRA, prefix-tuning, token-ID mapping and instruction-data packing, and training-free optimization techniques such as chain-of-thought prompting, beam search and retrieval-augmented generation.

Further diagrams cover direct preference optimization compared with RLHF, reinforcement-learning fundamentals such as Markov decision processes, exploration-exploitation, Monte Carlo and temporal-difference methods, and DQN, and policy-optimization architectures and their derivatives, including actor-critic methods, generalized advantage estimation, PPO, TRPO, PPO-Clip, deterministic policy gradients and DDPG, continuing into RLHF and RLAIF.

Diagrams are provided as high-resolution images and as scalable SVG vector files; a companion book covering the same material in more depth is sold separately. It is written in Python.

Website: https://book.douban.com/subject/37331056/