FlagGems
A high-performance operator library for large language models, implemented in the Triton language, that registers with PyTorch's ATen backend to accelerate training and inference across many hardware platforms.
FlagGems is a generic, high-performance operator library for large language models, implemented in the Triton language on a set of backend-neutral kernels, aimed at accelerating LLM training and inference across more than ten different hardware platforms. It is part of FlagOS, a fully open-source system-software stack meant to unify the model, system, and chip layers so an AI workload can be developed once and run anywhere, reducing fragmentation between chipset-specific software stacks.
By registering with PyTorch's ATen backend, FlagGems lets model developers switch to Triton kernels without changing their existing low-level PyTorch APIs, so they keep using familiar PyTorch calls while benefiting from new hardware acceleration. It provides a large collection of PyTorch-compatible operators, hand-optimized performance for selected operators, eager-mode execution independent of torch.compile, automatic pointwise operator code generation for arbitrary input types and layouts, and fast per-function runtime kernel dispatching; a C++ Triton function dispatcher is in progress.
It is released under the Apache-2.0 license.