Bonsai Demo
A local demo application for running the Bonsai 2 27B language model, a heavily compressed reasoning model with vision and tool-calling support, on a laptop or single GPU.
Bonsai Demo runs the Bonsai 2 27B language model locally on a Mac using Metal, on Linux or Windows using CUDA, Vulkan or ROCm, or on CPU. Bonsai 2 27B packs full 27-billion-parameter-class reasoning into ternary weights at 5.9GB, reported to retain most of a full-precision model's intelligence at roughly a ninth of the size, with math performance close to full precision and coding at the same baseline level.
It supports vision input, so photos, screenshots and PDFs can be sent and asked about, native OpenAI-style tool calling with full round-trips and MCP servers in both of the demo's user interfaces, a 262,000-token context kept practical on-device through a hybrid-attention architecture, and adjustable reasoning effort per chat or per request. It requires this demo's own fork of llama.cpp binaries, which a setup script downloads automatically, since the stock llama.cpp cannot run Bonsai's packed weight format; running the setup script and then a server-start script gives a chat, vision and tools interface at a local address.
It is a Shell-scripted demo released under the Apache 2.0 license.