LocalAI
LocalAI is an open-source AI engine that runs language, vision, voice, image and video models on any hardware with no GPU required, behind drop-in OpenAI, Anthropic and ElevenLabs-compatible APIs.
Each backend, such as llama.cpp, vLLM, whisper.cpp or stable-diffusion, ships in its own image and is pulled only when a model actually needs it, so a deployment installs nothing unused; supported hardware includes NVIDIA, AMD, Intel, Apple Silicon and Vulkan GPUs as well as CPU-only machines. It supports multiple users with API-key authentication and per-user quotas, and includes built-in autonomous agents with tool use, RAG, MCP and skills.
All processing runs on the operator's own infrastructure with no telemetry by design. A built-in terminal agent, started alongside a running server, can answer questions, read files and run commands, asking for approval before anything that changes state, and also supports fine-tuning and quantization workflows.
Go, MIT licensed.