MLX LM
A Python package for generating text with and fine-tuning large language models on Apple silicon using Apple's MLX framework.
MLX LM is a Python package for generating text and fine-tuning large language models on Apple silicon with MLX. It integrates with the Hugging Face Hub to use thousands of LLMs with a single command, supports quantizing models and uploading them back to the Hub, and supports low-rank and full model fine-tuning, including of quantized models, plus distributed inference and fine-tuning across machines.
It installs via pip or conda, and provides command-line tools to generate text from a prompt or open an interactive chat REPL that preserves context for the session's lifetime, with command-line options for choosing a model and sampling parameters. Its default model for generation and chat is a 4-bit Llama-3.2-3B-Instruct build, and any MLX-compatible model, including the thousands available in the MLX Community Hugging Face organization, can be selected instead.
A Python API exposes load and generate functions for scripting, and a convert function quantizes a model and optionally uploads the result to a chosen Hugging Face repository.