MimiModel
A from-scratch C inference engine that runs a 45-million-parameter tool-calling language model entirely on a five-dollar ESP32-S3 microcontroller, with no Linux, Python or network connection.
MimiModel is a single-file C99 inference engine, about 2,000 lines with no dependencies beyond libm, that runs Cactus Compute's Needle 2 model for tool calling, device control and structured extraction directly on an ESP32-S3 microcontroller. The model's 13.7MB of weights live in flash and are memory-mapped rather than loaded into RAM.
On the ESP32-S3 it reaches about 2.1 tokens per second prefill and 1.7 tokens per second decode in its fixed one-tool configuration, and scores 69.6% on a mobile-actions benchmark of 961 cases, matching the official Needle engine's accuracy on the same inputs while running several times slower than a cloud API. A host CLI, installed from source with pip, keeps one serial connection open to the board so it can reuse the model's KV prefix cache between commands; building and flashing the firmware requires ESP-IDF v5.5 or newer.