Lumen
A single-binary LLM inference engine written in Rust for Apple Silicon and NVIDIA CUDA, built from scratch with native CUDA and Metal kernels, a native tokenizer and its own model format.
Lumen downloads a model, converts it, and runs it GPU-resident with one command, without depending on PyTorch, ONNX or Python. CUDA kernels JIT-compile at runtime through NVRTC, so a single CUDA build runs on any compute-capability-8.0-or-later device without needing a build-time CUDA SDK.
It is production-ready for the shipped Qwen3.5 and Qwen3.8 model family, including dense 9B and 27B models and a 35B mixture-of-experts model, on Apple Silicon and NVIDIA CUDA; the public API and its own .lbc model format are not yet stable.
It installs from a prebuilt binary that detects the platform automatically, or by building from source with Cargo.