mlex
A dependency-free Rust runtime for Apple's MLX framework that loads Hugging Face model checkpoints straight onto Apple Silicon GPUs for fast local inference, shipped as both a native Rust crate and a Node.js package.
mlex is a from-scratch Rust implementation of Apple's MLX inference stack with no Python interpreter, system MLX install or PyTorch dependency: pointed at a Hugging Face model directory, it loads the model onto the GPU through Metal and starts generating, built on MLX's Metal backend, the same engine behind mlx-lm, with mlx and mlx-c vendored and statically linked, and distributed as a native Rust crate and a Node.js package built from the same core.
It supports Qwen2, Qwen3, Qwen3.5 (dense, mixture-of-experts and vision variants), Gemma4, NemotronH, DharaAR and any vanilla-Llama-shaped checkpoint; it reads a checkpoint's quantization configuration directly rather than relying on a hardcoded list, covering dense bf16/fp16, affine 2- to 8-bit quantization at any group size, mxfp4, mxfp8, nvfp4 and mixed per-layer precision formats. It accepts image input on vision-capable checkpoints and audio or sampled video frames alongside text on Gemma4 checkpoints with the matching towers; supports system prompts in the OpenAI/Anthropic style, opt-in reasoning mode with a token budget, OpenAI-style tool calling with both Hermes-style and Gemma-native parsing, and stateless prompt caching that transparently reuses KV state for a repeated prefix across calls.
Built with Rust and Node.js. MIT licensed.