tiny-llm
A minimal, hand-written LLM inference engine for the MiniCPM5-1B model, built in Rust to study how transformer inference works, with a terminal UI that visualizes generation as it happens.
tiny-llm is a minimal inference engine for the MiniCPM5-1B model, written purely in Rust as a learning project to study how large language models work at the level of individual tensor operations. It implements token embedding lookup, rotary position embeddings, grouped-query attention, a SwiGLU MLP and RMSNorm by hand, with a simple key-value cache that speeds up generation up to sixfold.
A terminal UI shows a real-time view of how the model is generating while it runs on CPU only, with no GPU required. Separate branches keep a version built on the candle tensor library and a version without the key-value cache, for comparison. It is MIT licensed.