EasyR1
An efficient, scalable, multi-modality reinforcement-learning training framework for language and vision-language models, forked from veRL to add vision-language support.
EasyR1 is an efficient, scalable, multi-modality RL training framework, built as a fork of the veRL project extended to support vision-language models. It is efficient and scalable due to the design of its HybridEngine and the SPMD mode of the latest vLLM release.
It supports Llama3, Qwen2, Qwen2.5, and Qwen3 language models, Qwen2-VL, Qwen2.5-VL, and Qwen3-VL vision-language models, and DeepSeek-R1 distilled models, along with algorithms including GRPO, DAPO, Reinforce++, ReMax, RLOO, GSPO, and CISPO, on any text or vision-text dataset in a supported format. It supports padding-free and LoRA training, resuming from the latest or best checkpoint, and tracking through Wandb, SwanLab, MLflow, or TensorBoard, and it is used by Amazon Web Services. It provides a Dockerfile and a pre-built Docker image, and can run inside Apptainer when Docker is unavailable, and models can be downloaded from the ModelScope hub as an alternative source.