microjpt
A single-file, dependency-free Julia port of Karpathy's microgpt that trains and runs a small GPT model from scratch, written to explore Julia's numerical performance.
microjpt is a pure Julia port of Karpathy's microgpt, a single-file, zero-dependency GPT implementation that trains and generates text from scratch using native matrix operations and manual backpropagation rather than an autograd framework. It exists to explore Julia's expressiveness and performance for machine learning code, and ships as two files: a fully commented version with every operation annotated, and a minimal, uncommented version of the same algorithm.
Both files implement a complete pipeline: a tokenizer, RMSNorm, multi-head causal attention, a ReLU multilayer perceptron, manual matrix backpropagation, an Adam optimizer, and temperature-controlled inference. Independent benchmarks on an Apple M5 report it as the fastest batch-size-one implementation among several compared Python, C++, and Rust ports of the same algorithm, attributing the gap to collapsing tens of thousands of autograd operations into a small number of explicit matrix operations.
It installs by installing Julia through juliaup and running the scripts directly.