MLX-LM-LoRA
A training toolkit for fine-tuning large language models locally on Apple Silicon with MLX, supporting LoRA, DoRA, full-precision and quantized training across many preference-optimization algorithms.
MLX-LM-LoRA trains any model supported by MLX-LM, including Llama, Mistral, Qwen, Gemma, OLMo and MiniCPM, using LoRA, DoRA, full-precision training, or quantized QLoRA training at 4, 6 or 8 bits. Quantization-aware training applies a quantization projection during supervised fine-tuning, DPO or ORPO training so the model better matches its eventual quantized form.
Supported training algorithms include supervised fine-tuning, DPO, CPO, ORPO, GRPO, GSPO, Dr. GRPO, DAPO, online DPO, XPO, KL-regularized RLHF and PPO, plus a method for repairing repetition loops. A custom preference model can also be trained for online preference training. Example notebooks and configuration files for each method live in a separate companion repository.