afm-server
A macOS menu bar app that exposes Apple's on-device Foundation Models through OpenAI- and Ollama-compatible local API endpoints, so nothing leaves the machine.
afm-server is a macOS menu bar application that bridges Apple Intelligence's on-device Foundation Models with OpenAI- and Ollama-compatible API endpoints, letting existing tools that speak those APIs run against a local, on-device model instead of a cloud service. It streams tokens over Server-Sent Events, keeps multi-turn conversation context in cached sessions with a 30-minute timeout and up to 50 concurrent sessions, and limits concurrent inference with a configurable, FIFO-queued semaphore.
It automatically evicts a session after a safety violation so one bad message cannot poison the rest of a conversation, and separately detects when the model refuses as ordinary text rather than an exception, resetting the session so later messages keep working. It supports OpenAI-compatible delegated tool calls for agent harnesses, plus its own local file and terminal tools, and checks daily for signed, notarized updates. It requires macOS 26, an Apple Silicon Mac, and Apple Intelligence enabled.