ESP32-S3 Distributed AI
A distributed micro-language-model system that splits a 56 million parameter model across three ESP32-S3 microcontrollers connected over ESP-NOW, with a browser interface for typing prompts and streaming generated text.
ESP32-S3 Distributed AI runs a 56-million-parameter language model, trained on the WikiText-103 corpus using Google's Gemma-style per-layer embeddings and quantized to 4 bits, split across three ESP32-S3 N16R8 boards that communicate wirelessly over ESP-NOW. The 50.3-million-parameter embedding table is split across two of the boards to fit within each board's 16 MB of flash, and the third board maintains a key-value cache in PSRAM so the model can attend to the full generated sequence instead of only the most recent token.
A user types a prompt in a browser connected to one board's Wi-Fi access point; that board forwards the prompt over ESP-NOW to the tokenizing board, which computes a partial embedding projection and requests the remaining embedding row from the third board, combines them, and sends the result to the board running six transformer layers of attention and feed-forward computation; the output flows back through the chain and streams to the browser over server-sent events as it is generated.
The project extends an earlier single-board demonstration to a multi-board distributed architecture with web-based interaction, and is implemented in embedded C.