Back to directory

Airframe

A pure-Rust inference engine that runs GGUF language models directly on the GPU through WebGPU compute shaders, working across NVIDIA, AMD, Intel, integrated, and Apple Silicon GPUs without a C++ toolchain or llama.cpp.

Airframe

Airframe runs full transformer inference on the GPU via WGSL compute shaders, avoiding the C++ toolchain, native library linking, and cross-compilation difficulty that come with wrapping llama.cpp. It builds with plain cargo build and targets WebGPU (wgpu) as its backend, so the same code runs on NVIDIA, AMD, Intel, integrated GPUs, and Apple Silicon.

It is the GPU inference core behind the Shimmy project, with a certification pipeline covering math, inference, and determinism across dozens of model and quantization combinations, and metadata-driven architecture support derived from GGUF tensor presence rather than hardcoded architecture strings.

The WebGPU inference runtime -- attention, the GGUF loader, and quantization -- is MIT licensed. A separate Fused Semantic Execution subsystem included in the repository is covered by a pending patent and licensed differently.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.