VulkanForge
An LLM inference engine for AMD RDNA4 GPUs written in Rust using Vulkan compute shaders, supporting GGUF models and native FP8 matrix operations.
VulkanForge is an LLM inference engine for AMD RDNA4 GPUs, built in pure Rust with Vulkan compute shaders instead of a vendor-specific GPU runtime. It compiles to a roughly 14MB static binary with no dependency beyond the system Vulkan loader, and the project describes it as the first engine to do native FP8 matrix-core computation over Vulkan on consumer AMD hardware. It builds on an earlier ROCmForge project for its model loader, CPU inference path, GGUF parser and overall architecture.
An optional, off-by-default server mode can host a persistent, project-scoped semantic memory inside the API process, gated behind both a build feature flag and a runtime flag. It exposes endpoints to remember, recall, archive, supersede, derive and mark notes as contradicting each other, backed by a combined graph-and-vector SQLite store with a local CPU embedding model.