Back to directory

Deltafin

A single native binary that runs the full, unpruned 2.8-trillion-parameter Kimi K3 model on consumer hardware, using small draft models only to speed up generation, never to reduce quality.

Deltafin

Deltafin runs the complete Kimi K3 model, all 16 routed experts and every token, on consumer hardware rather than the roughly 16-node, multi-terabyte-VRAM infrastructure Moonshot recommends for it. Its stated rule is that speed must never come from reducing model quality: small draft models are allowed to guess ahead, which is where much of its speed comes from, but K3 itself checks every guess and nothing reaches the user without K3's own approval.

The project frames itself as research into how far consumer hardware can be pushed to run a model of this size, rather than a finished product, and publishes ongoing throughput benchmarks on an M1 Max laptop as it improves. It targets machines on the order of a high-end workstation rather than typical consumer laptops.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.