MiniVLLM
A from-scratch reimplementation of the vLLM inference engine with self-contained paged and flash attention, built to benchmark attention mechanisms during prefill and decode.
MiniVLLM reimplements the core of the vLLM inference engine, based on Nano-vLLM, with its own paged attention and flash attention rather than relying on outside attention libraries. A companion step-by-step guide walks through the implementation, covering layers, models, paged attention, CUDA graphs and scheduling.
Its main demo runs a small, randomly initialized Qwen3 model through the custom engine with batch processing, paged attention and KV cache management, generating text for sixty repeated chat prompts. Two further scripts benchmark the prefilling phase, comparing standard PyTorch attention, a naive Triton kernel and flash attention by memory use, and the decoding phase, comparing naive PyTorch, a vectorized PyTorch implementation and a custom Triton kernel for paged attention.
The project runs with the uv package manager and can scale to multiple GPUs by changing a world-size setting in its configuration. It is written in Python and released under the Apache-2.0 license.