Paddock
A native Rust inference server for running open models on NVIDIA GPUs and Apple Silicon, with OpenAI- and Anthropic-compatible APIs and a bundled Studio for comparing models.
Paddock is an AI inference server written in Rust for NVIDIA GPUs and Apple Silicon. It exposes an OpenAI- and Anthropic-compatible API and includes a Studio for exploring and comparing models, including cloud models reached through OpenRouter or other endpoints. The scheduler, paged KV cache, memory management, and the CUDA and Metal kernels are all implemented in the project itself rather than wrapped from another engine.
The project ships two binaries: paddock-runner, which runs models and is the only part strictly required, and paddock, a manager that bundles the Studio for a simpler setup. On NVIDIA systems only the vendor driver is needed. A native macOS app, written in Swift, combines the manager and Studio in one window and uses a Metal backend; it is a pre-release, still under active development, with visual glitches and incomplete features expected.
Downloads include a disk-image native app, a command-line installer package, and a plain archive of the command-line tools; the command-line build is shared between the NVIDIA Windows and Linux builds and the Apple Silicon build.