goinfer
A pure-Go, no-cgo local LLM inference engine distributed as a single static binary, running open-weight models such as Gemma, Qwen, and Llama from safetensors or GGUF with schema-constrained structured output.
goinfer runs open-weight large language models in pure Go, with no C bindings, no Python, no llama.cpp, and no CUDA toolkit required at runtime, packaged as one cgo-free static binary that boots in well under a second for a small model. It supports 35 model families loaded from either safetensors or GGUF files, checked for parity against their Hugging Face reference implementations, and supports schema-constrained structured output.
It installs either as a prebuilt binary download with the correct GPU backend already built in (Metal on macOS, CUDA on Linux), or built from source with the Go toolchain, where the default build produces a CPU-only server and GPU support requires building a separate backend-specific entry point. It can also be used as a Go library by importing its specific packages, such as decoder, tokenizer, or chat, rather than the bare module.