llama-vulkan-strix
Docker Compose configurations for serving llama.cpp GGUF language models on AMD Strix Halo hardware, covering stock Vulkan and custom ROCm builds with measured prefill, decode and quality metrics.
llama-vulkan-strix packages llama.cpp as Docker Compose services tuned for AMD's Strix Halo GPU architecture, serving a chosen GGUF model, such as members of the Qwen and Gemma families, on a local port with configurable context length and concurrent request slots set per model in an environment file. A stock Vulkan-only compose file works out of the box, while separate compose files add custom ROCm builds with newer quantization formats and multi-token prediction that the stock image cannot load.
Its README documents prefill and decode throughput, and quality-versus-original metrics such as KL divergence and refusal rates, all measured on the author's own hardware, alongside model-card capability scores, so a user can pick a model package based on measured tradeoffs between speed and quality rather than guessing.