MLX-Node
A toolkit for running MLX models from Node.js, covering language and multimodal inference, an HTTP server, model conversion, and GRPO/SFT training, with a Rust and C++ native layer and no Python required.
MLX-Node runs MLX machine learning models from Node.js: language and multimodal model inference, an HTTP server exposing an OpenAI-compatible responses endpoint, model conversion, and GRPO/SFT training for supported models. Its public API is TypeScript, with a native layer written in Rust and C++, and Python is not required at runtime; it targets Apple Silicon with Metal as its main platform, with an experimental CUDA backend for Linux on NVIDIA GB10/DGX Spark hardware.
Its CLI downloads models into a local cache, serves them over HTTP, and can also launch a coding agent against a locally served model. A separate library package exposes the language-model API directly, centered on a chat-session object that owns the conversation and KV cache and supports regular turns, streaming, tool results, and resetting the session. Supported model families include Qwen3 and Qwen3.5/3.6, dense and mixture-of-experts, with GRPO/SFT training support, Gemma4, LFM2/LFM2.5, and Nemotron 3.5 Lightning for general use, plus vision models, including OCR and document processing, an audio model for offline, streaming, and meeting transcription, and an embedding model.