# tiny-llm

A minimal, hand-written LLM inference engine for the MiniCPM5-1B model, built in Rust to study how transformer inference works, with a terminal UI that visualizes generation as it happens.

tiny-llm is a minimal inference engine for the MiniCPM5-1B model, written purely in Rust as a learning project to study how large language models work at the level of individual tensor operations. It implements token embedding lookup, rotary position embeddings, grouped-query attention, a SwiGLU MLP and RMSNorm by hand, with a simple key-value cache that speeds up generation up to sixfold.

A terminal UI shows a real-time view of how the model is generating while it runs on CPU only, with no GPU required. Separate branches keep a version built on the candle tensor library and a version without the key-value cache, for comparison. It is MIT licensed.

Website: https://github.com/AspadaX/tiny-llm