Cardputer AI
A fully offline chatbot that runs entirely on an ESP32-S3 Cardputer microcontroller, with an 8-million-parameter model quantized to 4 bits and baked into the firmware.
Cardputer AI is a chatbot that runs entirely on a Cardputer ADV, a pocket-sized microcontroller device with 512 KB of RAM and no internet connection. Its language model has 8 million parameters, quantized to 4 bits, stored as a 5.7 MB file baked directly into the firmware, and it generates about 5 tokens per second.
Unlike earlier tiny on-microcontroller language models, which could only continue a story once given an opening line, this one holds a back-and-forth conversation: it stays on topic, asks follow-up questions, and says it does not know an answer rather than inventing one. It also supports a separate story-writing mode that continues a prompt as a short narrative.
The firmware is downloadable and requires no cloud connection, no phone and no companion app.