Hawk Point NPU LLM
An experimental project for running small transformer checkpoints such as SmolLM 135M and Qwen2.5 0.5B directly on the first-generation AMD XDNA NPU in Ryzen AI processors, using MLIR-AIE kernels, a patched Ollama build and a native Colibri backend.
This project runs compatible small transformer checkpoints on the NPU of first-generation AMD XDNA Ryzen AI processors rather than the GPU, using MLIR-AIE and IRON to target the npu1 AIE2 array directly. The CPU handles tokenization, orchestration, argmax and SmolLM's large final head, while decoder projections, RMSNorm, RoPE, attention, KV caches and SwiGLU run as AIE2 kernels on the NPU, with Qwen's head running on the NPU except under a hybrid placement that moves it and the trailing layers to the CPU. It is an unaffiliated, experimental research effort, explicitly not affiliated with or supported by AMD, Xilinx, Hugging Face or Open WebUI.
Besides the native MLIR-AIE runtime with its own OpenAI-compatible server for small supported checkpoints, the repository includes a patch that adds XDNA1 NPU support to an upstream Ollama build so existing Ollama Qwen models can run across CPU, GPU and NPU, and a native C++/XRT implementation of the Colibri backend's ABI for routed-expert and decode matmul calls made directly from Colibri's C engine.
Built with Python and C/C++, with build and verification scripts for each runtime path; alpha status, not production-ready.