TRL
A library for post-training foundation language models with reinforcement learning and preference optimization methods, such as supervised fine-tuning, GRPO and DPO, built on top of Hugging Face Transformers.
TRL is a library for post-training foundation models using methods such as supervised fine-tuning (SFT), Group Relative Policy Optimization (GRPO) and Direct Preference Optimization (DPO). It is built on the Hugging Face Transformers ecosystem and supports a range of model architectures and modalities.
Dedicated trainer classes, including SFTTrainer, GRPOTrainer, DPOTrainer and KTOTrainer, wrap the Transformers trainer and natively support distributed training methods such as DDP, DeepSpeed ZeRO and FSDP, scaling from a single GPU to multi-node clusters through Hugging Face Accelerate. Integration with PEFT enables training large models on modest hardware through quantization and LoRA or QLoRA, and integration with Unsloth accelerates training with optimized kernels.
A command-line interface lets a user fine-tune a model without writing code. TRL installs from PyPI and is written in Python.