LLM Knowledge and Interview Questions
A personal, community-sourced collection of study notes and real interview questions about large language models, covering fundamentals, architecture, training, fine-tuning, inference, RLHF, retrieval-augmented generation and evaluation.
This repository collects notes and real interview questions about large language models, written up by its maintainer from public resources and organized for study rather than as an official course. Topics include language model fundamentals and history, tokenization and word vectors, Transformer architecture and attention variants, decoding strategies, and common large models such as LLaMA and ChatGLM.
Further sections cover training datasets, distributed training strategies, supervised fine-tuning methods including LoRA, inference frameworks and quantization, reinforcement learning and RLHF techniques such as PPO and DPO, retrieval-augmented generation and agents, and methods for evaluating large models and their hallucinations. It also links to companion hands-on projects that build a small Chinese language model, a retrieval-augmented generation system and an MCP server from scratch, plus a from-scratch reimplementation of Llama 3.