Shimmy
A single-binary, OpenAI-API-compatible local inference server for GGUF models, built entirely in Rust on the project's own WebGPU transformer engine, Airframe.
Shimmy is a single-binary inference server for GGUF models that speaks the OpenAI API, so existing AI tools can point at it and run locally and privately with no separate Python runtime or C++ toolchain. Under the hood it runs on Airframe, a Rust WebGPU (WGSL) transformer engine.
It targets deterministic output (the same model, seed and parameters produce the same result) through F32 accumulation precision, uses WGSL compute shaders that run on NVIDIA, AMD, Intel and integrated GPUs as well as Apple Silicon, derives a model's spec automatically from its GGUF metadata instead of hardcoding per-model constants, and supports YaRN RoPE scaling for extended context. Twenty-six model and quantization combinations across twelve model families pass its certification regimen covering math, inference and determinism.
It is written in Rust and licensed under Apache-2.0.