ZINC
A single Zig binary for running local large-language-model inference on AMD GPUs and Apple Silicon, loading GGUF files behind a command line, a browser chat interface, a model manager and an OpenAI-compatible API.
ZINC is an inference engine written in Zig that loads GGUF model files and runs them locally, without requiring Python or a CUDA-specific toolchain, exposing a command line, a browser-based chat interface, a model manager, and an OpenAI-compatible API. It targets AMD GPUs through a ROCm backend and a Vulkan backend, as well as Apple Silicon.
The project publishes benchmarks comparing its ROCm backend against llama.cpp's ROCm backend on the same GPU and the same GGUF files, reporting an advantage in both prompt processing and token generation across the models tested, and a separate benchmark for its Vulkan backend against llama.cpp's Vulkan backend. One tested model can use an extra prediction block it ships with to draft tokens that the full model then verifies in a single batched pass, which the project reports roughly doubles its generation speed for that model; the published comparison chart shows both engines without that feature enabled, for a like-for-like comparison. Benchmark numbers come from a script in the repository and are recorded with their prompts, raw samples and build revisions rather than hand-edited.
It requires Zig 0.15.2 or newer to build.