NANOLLAMA
A from-scratch training framework for Llama 3 architecture language models, covering data preparation, pretraining, LoRA personality fine-tuning, GGUF export and a small standalone Go inference engine.
nanollama trains a Llama 3-style model from raw text rather than fine-tuning or adapter-wrapping existing weights, using FineWeb-Edu data preparation, a Python pretraining loop with the Muon optimizer, and a LoRA personality fine-tuning stage that produces per-voice adapters at rank 64. A gamma-extraction step isolates the difference between a fine-tuned personality's weights and the base model's weights, and the result exports to a llama.cpp-compatible GGUF v3 file.
Eight model tiers are defined, from an 89 million-parameter English-only nano model up to a 7.9 billion-parameter model covering thirteen languages, using grouped-query attention above the mini tier and a progressively larger multilingual tokenizer at the larger tiers. A standalone, dependency-free Go inference engine of about nine megabytes runs the exported GGUF model. The project was originally forked from nanochat, with the earlier code kept in a legacy folder.