# llama-vulkan-strix

Docker Compose configurations for serving llama.cpp GGUF language models on AMD Strix Halo hardware, covering stock Vulkan and custom ROCm builds with measured prefill, decode and quality metrics.

llama-vulkan-strix packages llama.cpp as Docker Compose services tuned for AMD's Strix Halo GPU architecture, serving a chosen GGUF model, such as members of the Qwen and Gemma families, on a local port with configurable context length and concurrent request slots set per model in an environment file. A stock Vulkan-only compose file works out of the box, while separate compose files add custom ROCm builds with newer quantization formats and multi-token prediction that the stock image cannot load.

Its README documents prefill and decode throughput, and quality-versus-original metrics such as KL divergence and refusal rates, all measured on the author's own hardware, alongside model-card capability scores, so a user can pick a model package based on measured tradeoffs between speed and quality rather than guessing.

Website: https://github.com/hec-ovi/llama-vulkan-strix