How LLMs Work
How LLMs Work is an interactive local app that walks through the whole LLM pipeline hands-on, from a rule-based chatbot to training a small transformer from scratch, with runnable code for every stage.
How LLMs Work is a locally run interactive application covering five stages of how language models are built, each with its own working code. It starts with an ELIZA-style pattern-matching chatbot streamed word by word, then a small neural network trained live on the XOR problem to show why backpropagation was needed, then a byte-pair-encoding tokenizer trained from scratch on the user's own text with an animated view of its merge steps.
The next stage trains word embeddings from scratch with Word2Vec skip-gram and negative sampling, and the last trains a full decoder-only transformer from scratch with no machine-learning libraries: multi-head causal self-attention, layer normalization, feed-forward layers, backpropagation and Adam optimization are all implemented by hand, using multi-threaded data parallelism.
It is built with Hono for the server and JSX rendering and Vite for bundling, in TypeScript, and runs locally with pnpm and Node.js 20 or later.