ziggy-llm
A GGUF language model inference engine written from scratch in Zig for Apple Silicon, with its own quantization kernels, Metal GPU shaders and parser rather than wrapping an existing runtime.
ziggy-llm is a clean-sheet inference engine for GGUF language models on Apple Silicon, written entirely in Zig with zero external dependencies: its quantization kernels, Metal GPU shaders, GGUF file parser and inference runtime are all original code rather than a port of or wrapper around llama.cpp or any other engine. It deliberately targets only Apple Metal and the GGUF format, aiming for one thing done well rather than broad support, and ships as a single CLI binary.
The project publishes decode-throughput comparisons on an Apple Silicon MacBook Pro against llama.cpp and another Zig-based inference engine across several small and mid-sized open models, and describes a small OpenAI-compatible server as planned but not yet shipped. It requires the Zig 0.15.2 toolchain or newer to build.