# gmlx

gmlx is a local inference platform for running K-quant and IQ-quant GGUF models on Apple Silicon, serving them over OpenAI- and Anthropic-compatible APIs, with terminal or browser chat, voice input, a local RAG stack and LoRA fine-tuning.

gmlx runs the community's K-quant and IQ-quant GGUF model builds unchanged, using Metal kernels from a companion project to run them natively on Apple's MLX framework, and the project reports benchmarking faster than llama.cpp on the same file, with the largest gap at the long context lengths coding agents and long sessions use. A mixture-of-experts model larger than available RAM can still run by streaming its experts from disk.

A setup wizard writes a configuration file and can scan folders where models are already kept, after which a single command downloads a model under a short id that every other command then accepts in place of a full path; any GGUF file can also be run directly by its path with no configuration at all. The documentation includes a specific migration guide covering what carries over from llama.cpp, Ollama and LM Studio.

It requires an Apple Silicon Mac running macOS 26.2 or newer and installs through Homebrew or uv.

Website: https://asher.github.io/gmlx/