# ROLL

An efficient, user-friendly reinforcement learning library for training large language models at scale, covering preference alignment, complex reasoning and multi-turn agentic interaction.

ROLL is a reinforcement learning library for large language models designed to make efficient use of large-scale GPU resources. It targets human preference alignment, complex reasoning and multi-turn agentic interaction as its main use cases.

It uses a multi-role distributed architecture built on Ray for flexible resource allocation and heterogeneous task scheduling, and integrates technologies such as Megatron-Core, SGLang and vLLM to accelerate both training and inference. Recent releases add support for Qwen3.5 dense and mixture-of-experts models, on-policy distillation, an FSDP2 training strategy, Megatron combined with LoRA, partial GPU overlapping, and support for Ascend NPUs.

It is written in Python and released under the Apache 2.0 license.

Website: https://alibaba.github.io/ROLL/