Open R1
An open effort to reproduce DeepSeek-R1, providing scripts to train reasoning models with SFT and GRPO, generate synthetic training data, and evaluate the results.
Open R1 is a fully open reproduction of DeepSeek-R1, building the missing pieces of its training pipeline so others can reproduce and build on it. It mainly consists of scripts for training and generating synthetic data: a GRPO trainer, a supervised fine-tuning script, and a data-generation script that uses Distilabel to generate synthetic data from a model, along with a Makefile that runs each pipeline step.
The project follows the DeepSeek-R1 technical report in three steps: replicating the R1-Distill models by distilling a high-quality corpus from DeepSeek-R1, replicating the pure reinforcement-learning pipeline used to create R1-Zero with newly curated math, reasoning and code datasets, and showing that a base model can reach similar results through multi-stage training. Released artifacts include the Mixture-of-Thoughts reasoning dataset, distilled from R1 across mathematics, coding and science, and the CodeForces-CoTs dataset of competitive programming problems and solutions.
It is written in Python and licensed Apache-2.0.