bitnet.c
A minimal, zero-dependency LLM inference engine in pure C11 for GGUF models, with NEON and AVX2 SIMD CPU backends, mixture-of-experts routing, TurboQuant KV-cache compression, and a WebAssembly build.
bitnet.c loads GGUF models for dense, mixture-of-experts and hybrid state-space or attention architectures, splitting model anatomy, quant formats, backend state, transformer planning, CPU execution, GPU op emission, KV-cache helpers and prefill into separate modules rather than a single flat transformer loop. It supports quantized CPU inference across formats including I2_S, TQ1_0, TQ2_0, several Q4 through Q8 variants, IQ2 through IQ4, F16, BF16 and F32, with scalar, ARM NEON, x86 AVX2, AVX512 and WASM SIMD backends where kernels exist, plus optional Metal and WebGPU acceleration.
Other features include mixture-of-experts routing with mmap, pread and an expert LRU cache, a prompt cache, stop strings, logprobs, chat formatting, server-sent-event streaming, batch prefill, FP16 KV storage and TurboQuant KV-cache compression. CUDA is named as a planned target rather than an implemented backend, and the project builds with a plain Makefile.