AMD Strix Halo (gfx1151) — vLLM Toolbox/Container
This project packages vLLM for serving large language models on AMD Ryzen AI Max "Strix Halo" hardware as an Ubuntu-based, Toolbx-compatible Docker or Podman container tracking stable ROCm and vLLM releases.
This repository provides an Ubuntu 24.04-based container, compatible with Toolbx, Distrobox, Podman or Docker, for serving large language models with vLLM on AMD Ryzen AI Max "Strix Halo" (gfx1151) hardware. The image tracks the stable ROCm release and the latest stable vLLM release at build time, with a separate development channel for newer, less tested builds.
A companion tool, AI Toolbox Cockpit, is the recommended way to install, launch and update the container, providing tested profiles and unified-memory configuration guidance. The image has been tested for single-host vLLM serving and for two-host tensor-parallel serving over RCCL, both over Ethernet and over RDMA/RoCE, though performance benchmarks specific to this Ubuntu image have not yet been published.