# Qwisp

A single-model inference server for Qwen3.6-35B-A3B on Apple Silicon, with an OpenAI-compatible HTTP API, a custom Metal decode engine built outside MLX's op-graph, and an optional bit-for-bit lossless decoding mode.

Qwisp serves one specific model, Qwen3.6-35B-A3B, on Apple Silicon Macs through an OpenAI-compatible /v1/chat/completions and /v1/models API, plus a qwisp chat CLI. Its decode core, called Seedless, runs raw Metal command buffers outside MLX's normal op-graph using persistent buffers and int32 readback, while still relying on MLX for tensors, quantization and weight loading. A separate scheduling layer, Tell, drives speculative decoding.

Mixture-of-experts layers can stream from flash storage on machines with limited RAM, letting the 35B-parameter model run on smaller Macs. A strict decoding path reproduces the quantized model's greedy token stream bit-for-bit, and function calling follows the OpenAI tool-call format so agentic clients can use it directly. Reasoning output is separated from the final answer instead of leaking into the response content.

It requires an Apple Silicon Mac on macOS 14 or later and a roughly 20 GB model checkpoint, installable as a Homebrew binary or built from source with Xcode 26.

Website: https://github.com/penta2himajin/qwisp