MNN
A lightweight deep learning inference and training framework, used in production across more than 30 Alibaba apps, with companion projects for running large language and stable diffusion models locally on phones, PCs and IoT devices.
MNN is a lightweight deep learning framework supporting both inference and training, tuned for on-device performance. It is integrated into more than 30 Alibaba apps, including Taobao, Tmall, Youku, DingTalk and Xianyu, across more than 70 usage scenarios such as live broadcast, short video capture, search recommendation and image-based product search, and is also used on embedded and IoT devices.
MNN-LLM is a large language model runtime built on the MNN engine for deploying models such as Qwen, Baichuan, Zhipu and LLaMA locally on a phone, PC or IoT device, and MNN-Diffusion does the same for local stable diffusion image generation. Sample apps built on these runtimes include a multimodal chat app for Android and iOS and an offline avatar app that runs speech recognition, text-to-speech and a language model together on-device.
Inside Alibaba, MNN is the compute module of the Walle system, a device-cloud collaborative machine learning system described in a paper at the OSDI 2022 systems conference. It is written in C++ and released under the Apache 2.0 license.