local-kimi
An optimized local serving engine for the Kimi-Linear-48B model on a single consumer GPU, using INT4 quantization and fused decode kernels, with a protocol bridge that lets Claude Code, Codex, Cline, Aider and opencode connect to it unchanged.
local-kimi serves the Kimi-Linear-48B model within a 32 GiB memory cap on a single NVIDIA GPU such as an RTX 5090, using an INT4 quantizer and fused decode kernels that the project measures at 113.83 tokens per second, a 3.18 times speedup over an unoptimized baseline. A bundled bridge component, k3, detects which client is calling it from the request's route, headers, user agent and body, and translates between that client's protocol and the model's OpenAI-compatible Chat Completions backend, so Claude Code, Codex, Cline, Aider and opencode can all point at the same local server without reconfiguration.
The project documents k3's protocol coverage against vLLM, llama.cpp and Ollama in a comparison table, noting where its own behavior, such as translating tool calls to and from an OpenAI Chat upstream or restoring reasoning traces across client turns, is verified rather than only documented. It does not serve inference itself; it is a standalone adapter in front of an existing OpenAI Chat Completions backend, and is Apache-2.0 licensed.