FinetuneForge
A progressive, curriculum-style collection of scripts and notebooks for fine-tuning Hugging Face Transformers models, moving from the high-level Trainer API down to a hand-written training loop, tokenizer details, and methods such as LoRA and prefix tuning.
FinetuneForge organizes fine-tuning material as a progressive curriculum rather than a single quick-start script. Early sections cover fine-tuning with the Trainer class and swapping in TrainingArguments, custom datasets and metric integrations; later sections reconstruct a training loop by hand, without Trainer, covering optimizer steps, gradient clipping, and tokenizer details such as padding side, truncation and data collators.
A further section works through parameter-efficient fine-tuning, with practical implementations of LoRA and prefix tuning alongside an explanation of why each layer is frozen or trained. The material is aimed at cases where a fine-tuning run departs from a tutorial's toy example and needs to be understood at the level of gradient flow, tokenizer behavior and learning-rate scheduling rather than treated as a black box.
It is distributed as scripts and notebooks for Hugging Face Transformers, Accelerate, Datasets and Evaluate.