AndroLLM
An Android app that runs large language models on-device using Google's LiteRT-LM runtime, with GPU acceleration, cloud-provider integration, persistent memory and offline voice interaction.
AndroLLM is a production-grade AI platform for Android that runs local .litertlm models through Google's LiteRT-LM runtime, a pure-Kotlin engine with no native code, so models run entirely offline after download.
Inference uses the XNNPACK CPU backend on every arm64 device or an OpenCL-based GPU delegate, with automatic fallback to CPU and recovery from corrupted output. Multi-turn conversations use KV-cache persistence, and per-model chat templates and stop sequences come from container metadata.
The app can also connect to any LiteLLM-compatible cloud provider, adds persistent memory through vector embeddings and hybrid retrieval, and includes a fully offline voice assistant pipeline covering wake word detection, speech recognition, the LLM, and text-to-speech.