Atlas
Atlas is an open-source LLM inference engine written in pure Rust and CUDA that serves an OpenAI-compatible API from a single small binary, with hand-tuned kernels, quantization and speculative decoding.
Atlas is an LLM inference engine written in Rust and CUDA, distributed as a roughly 75MB binary with no Python or PyTorch dependency and no runtime compilation. It serves OpenAI-compatible, Anthropic-compatible and Responses API endpoints, using hand-tuned CUDA kernels for each combination of hardware, model and quantization it supports.
It implements NVFP4 and FP8 quantization, speculative decoding through draft heads and block diffusion, radix-tree prefix caching, and expert parallelism across multiple nodes. It is verified on NVIDIA's DGX Spark (GB10, Blackwell), and compiles the same CUDA source for AMD's Strix Halo (gfx1151) through SCALE. Every release image is tested against a serve gate covering boot, coherence, tool calls and throughput, and a release that serves slower than its baseline fails that gate.
It installs from a signed script that verifies its own checksum, from source with cargo install, or as a single Docker image bundling its whole supported model catalogue. Atlas is licensed under AGPL-3.0.