MiniMind-O
A roughly 0.1-billion-parameter end-to-end omni model, trained entirely from scratch, that takes text, speech, and image input and produces text and streaming speech output.
MiniMind-O is the third project in the MiniMind series, following a text-only language model and a vision-language model, and aims to be one of the smallest complete open implementations of an omni model: one that listens, sees, and speaks with a single set of weights, rather than chaining separate speech-to-text, language-model, and text-to-speech stages, which the project argues adds latency and loses tone and emotional information. It uses a Thinker-Talker architecture, where the Talker predicts multiple codec tokens at once and supports real-time interruption during streaming speech generation.
Speech and vision input are encoded with frozen SenseVoice-Small and SigLIP2 models and projected into MiniMind's hidden space, with audio represented using the Mimi codec. The project releases full training code, small and full training datasets, several built-in voices with voice cloning, a technical report, and a web demo, implementing its core algorithms directly in PyTorch. It is Apache 2.0 licensed.