yzma
yzma is a Go library that embeds llama.cpp directly into Go applications for local inference of language and vision-language models, with hardware acceleration and no CGo dependency.
yzma lets a Go program call llama.cpp for fully local inference of large and small language models and vision-language models, on Linux, macOS, Windows, or in a browser as WebAssembly. It uses the purego and ffi packages instead of CGo, so no C compiler is required to build against it.
It supports hardware acceleration through CUDA, Metal, OpenVINO, Vulkan or, in the browser, WebGPU. A companion command-line tool downloads prebuilt llama.cpp libraries for the current platform and verifies them against published SHA-256 digests, and applications can also fetch the libraries themselves with automatic CUDA and ROCm detection.
yzma tracks current llama.cpp releases, so applications built on it gain new model support, performance work and fixes as llama.cpp ships them.