halogen-flash-server
halogen-flash-server is a specialized inference server that runs the Qwen3.8-Flash-Next model on AMD Strix Halo (gfx1151) GPUs, with kernels tuned for that one GPU and model family instead of a general-purpose runtime.
halogen-flash-server is an inference server built to run one model family, Qwen3.8-Flash-Next, on one GPU, AMD's Strix Halo (gfx1151), as fast as possible. Every kernel is written specifically for this GPU and model combination, with no general-purpose runtime, portability layer or fallback path.
It reports measured prefill and decode throughput on a reference Ryzen AI Max+ 395 machine with 128GB of RAM and ROCm 7.14.0, including speculative decoding via a draft head and prompt lookup that is verified to produce output identical to serial greedy decoding at temperature 0. It documents its quantization precision, measured from the checkpoint's own tensor table, in docs/QUANT.md.
Since version 0.7.0 it can also load a llama.cpp GGUF file of the same model directly, using the same kernels, speculation and identity guarantee without a conversion step.