# How LLMs Work

How LLMs Work is an interactive local app that walks through the whole LLM pipeline hands-on, from a rule-based chatbot to training a small transformer from scratch, with runnable code for every stage.

How LLMs Work is a locally run interactive application covering five stages of how language models are built, each with its own working code. It starts with an ELIZA-style pattern-matching chatbot streamed word by word, then a small neural network trained live on the XOR problem to show why backpropagation was needed, then a byte-pair-encoding tokenizer trained from scratch on the user's own text with an animated view of its merge steps.

The next stage trains word embeddings from scratch with Word2Vec skip-gram and negative sampling, and the last trains a full decoder-only transformer from scratch with no machine-learning libraries: multi-head causal self-attention, layer normalization, feed-forward layers, backpropagation and Adam optimization are all implemented by hand, using multi-threaded data parallelism.

It is built with Hono for the server and JSX rendering and Vite for bundling, in TypeScript, and runs locally with pnpm and Node.js 20 or later.

Website: https://github.com/w3cj/how-llms-work