Strix Halo llama.cpp toolbox
A performance-tuned build of llama.cpp for the AMD Strix Halo APU, bundling Flash Attention and MoE-prefill fixes with a current Mesa driver to speed up long-context, quantized-KV inference.
This project packages llama.cpp with a set of Flash Attention and mixture-of-experts prefill fixes for AMD Strix Halo hardware (Ryzen AI Max+ 395, Radeon 8060S, gfx1151), aimed at long-context workloads using a quantized KV cache, alongside a bundled current GPU driver so that quantized KV is fast instead of a performance penalty. Both Vulkan and HIP backends are supported.
Benchmarks comparing the same box and session against stock upstream llama.cpp show gains such as 3.26x faster deep-context prefill on a Qwen3-Coder MoE model and roughly 1.85x on a dense Qwen2.5-7B model, with every reported number traced to a raw benchmark run and repeat stock measurements shown to be stable within 0.5% across weeks. It is distributed as a portable directory, a Docker image and a distrobox container.