# Qwen 3.8 27B ROCmFP4_FAST on AMD Strix Halo (Ryzen AI Max+ 395)

A memory-optimized ROCm build of Qwen 3.8 27B for AMD Strix Halo APUs, combining 4-bit block quantization, speculative decoding, and a KV-cache scheme to raise inference throughput on unified-memory hardware.

This repository packages Qwen 3.8 27B for AMD Strix Halo (Ryzen AI Max+ 395 / Radeon 8060S) APUs, targeting single-machine inference on 128 GB of unified memory. It combines a custom 4.26-bit block quantization format, multi-token-prediction speculative decoding, an asymmetric KV-cache quantization scheme, and a cooperative-matrix attention engine.

The GGUF model weights are published on Hugging Face, and the format requires a patched llama.cpp fork pinned to a specific commit; the standard llama.cpp or Ollama builds cannot load these files. The README documents the quantization rationale, benchmark numbers, context-scaling behavior, and build and troubleshooting steps for the required engine.

A related repository extends the same approach to other models and other AMD Radeon GPUs.

Website: https://github.com/julianmb/q38rocm