bitgpu
A dependency-free WebGPU runtime for running 1-bit, binary-weight large language models directly in the browser, with GPU-resident decoding, streaming output and cross-turn KV-cache reuse.
bitgpu runs low-bit language models in the browser using WebGPU, currently targeting 1-bit, binary-weight models such as several sizes of Bonsai built on the Qwen3 and Qwen3.5 architectures. Each model size is checked bit-exact or greedy-exact against a reference implementation, and weights stream once from the Hugging Face Hub so a page can load a model and let the user chat with it on their own GPU.
Decoding runs entirely on the GPU, with streaming output, an abort signal, cross-turn KV-cache reuse, optional compressed KV-cache formats for long contexts, conversation snapshots that survive a page reload, and an optional attention-sink window for unbounded chats in fixed memory. It installs as an ESM-only npm package with zero runtime dependencies, and its own benchmark shows faster prefill and decode than the equivalent path in transformers.js on the same weights and hardware.