Sipp
An SDK and WebGPU runtime, built in Rust and C++, for running GGUF language models locally in the browser, on desktop, or through a cloud gateway, behind one API.
Sipp is an SDK for running AI inference across local, provider, and cloud-gateway backends behind a single, symmetric API. Its core, Sipp Engine, runs GGUF models directly in the browser, on the desktop, or on cloud infrastructure.
The SDK lets an application split logic between local and cloud compute instead of maintaining separate web runtimes, native wrappers, and middleware for protecting API keys. It packages the WebGPU engine together with a container gateway proxy in one toolkit.
It is under active development, so interfaces can change between releases. Planned work includes embedded vector memory, on-device data masking, and automatic request routing.