LoFT CLI
A command-line toolkit for fine-tuning small language models with LoRA, quantizing them, and running them locally on CPU, designed for low-RAM laptops without a GPU.
LoFT's finetune command trains LoRA adapters on an instruction-tuning JSON dataset using the peft library, training only the added LoRA layers rather than the full model; merge folds the trained adapter into a standalone Hugging Face model, export converts that model to GGUF format, quantize produces a Q4_0 GGUF file, and chat runs the resulting model offline through llama.cpp.
It targets the case of a laptop and a small dataset rather than a cloud GPU, so a domain-specific adapter for a small one-to-three-billion-parameter model can be trained and run entirely offline, including on an 8GB MacBook with no GPU at all. The project's author has stopped further development as of 2025, though the existing code and its published benchmarks are described as still working. It is licensed MIT and written in Python.