fak
An open local runtime that combines native inference, speculative decoding and agentic caching so useful local AI agents run faster automatically, and can also guard an agent a user already runs elsewhere.
fak is an open runtime aimed at making useful local AI agents practical on a user's own machine. One command sizes a model and context budget from the machine's unified memory, holds back new work under memory pressure instead of swapping, and starts a local OpenAI-compatible endpoint along with an interactive chat REPL; a separate mode instead wraps an agent the user already runs elsewhere.
The project reports its own Metal engine reaching near parity with a pinned llama.cpp reference build on an Apple M3 Pro, with token-for-token identical output, though the documentation is explicit that automatic setup and cache reuse currently have model and backend limits, and that speculative decoding is not yet universally enabled. It works with OpenCode and Claude Code as agent front ends, and installs with a shell script or a Go toolchain command.