DwarfStar
A small, native local inference engine for running a curated set of large language models, including DeepSeek V4 and GLM, on consumer hardware such as Macs, NVIDIA GPUs and Strix Halo systems.
DwarfStar is a small, native inference engine focused on running a curated set of large language models on consumer hardware. It is optimized first for DeepSeek V4 Flash, including its experimental vision model, DeepSeek V4.1 Flash, GLM 5.2 and 5.3, GLM 5.3 Flash, DeepSeek V4 PRO, and Qwen3.8 Flash Next, using its own GGUF files rather than acting as a general GGUF runner.
It supports Metal on Macs with 96GB of memory or more, with SSD streaming for smaller machines or very large models; NVIDIA CUDA, including multi-GPU setups and older Ada Lovelace cards that other backends do not support; and ROCm on Strix Halo systems such as the Framework Desktop. Model loading, prompt rendering, tool calls, KV cache state, the HTTP server and the coding agent are built and tested together, and the repository also includes tooling and data for GGUF conversion, importance-matrix generation, and quality and speed measurement.
It is built on llama.cpp and GGML, written in C, and licensed under the MIT license.