OlliteRT
An Android app that turns a phone into an OpenAI-compatible LLM server, running models locally through Google's LiteRT-LM runtime on the phone's GPU or CPU and serving them over the local network with no cloud or subscription.
A model is downloaded with one tap, from Hugging Face, an imported .litertlm file, or a custom model source added as a JSON file or URL, then started as a server; the phone then answers standard OpenAI-compatible HTTP requests from any client that already speaks that API, including Home Assistant, Open WebUI, OpenClaw, Python or curl. Supported models include vision, audio and reasoning-capable models with streaming output and experimental tool calling.
A built-in benchmark compares models on the device's own hardware, and a live stats dashboard, an optional floating monitor, Prometheus metrics for Grafana, and a Home Assistant REST API cover monitoring and remote control. Configuration includes per-model inference settings, GPU or CPU accelerator choice, idle-model unload, listener scope, client IP access rules, bearer-token authentication, and auto-start on boot for low, continuous power draw.
Using it with Home Assistant for voice commands currently needs a custom integration such as Extended OpenAI Conversation alongside OpenAI STT.