ZML
A production inference stack that compiles a single model codebase to run on NVIDIA, AMD, Intel, TPU, or Trainium hardware without rewriting it for each accelerator.
ZML is an inference stack for running AI models across different hardware accelerators from one codebase. It compiles models to NVIDIA, AMD, Intel, Google TPU, and AWS Trainium/Inferentia targets, aiming for hardware performance without per-accelerator rewrites.
It ships example programs, including an MNIST classifier and a large-language-model runner that supports models such as Llama 3.1/3.2, Qwen 3.5, and LFM 2.5, loadable from Hugging Face, a local directory, or S3, usable for one-off prompts or an interactive chat loop.
It is built with the Zig language, MLIR, and Bazel, and depends only on Bazel (via Bazelisk) to build.