CodeRL
CodeRL is the official research code for a NeurIPS 2022 paper that trains a pretrained code-generation language model with deep reinforcement learning, using a critic network to judge functional correctness and guide a regeneration step at inference time.
CodeRL treats a code-generating language model as an actor network and introduces a critic network trained to predict whether a generated program is functionally correct, giving the actor dense feedback signals during training rather than only a final pass-or-fail result from unit tests. At inference time, a critical-sampling generation procedure lets the model automatically regenerate a program based on feedback from example unit tests and the critic's scores.
The repository provides the code for pretraining and fine-tuning CodeT5-based models with and without CodeRL, generating programs, running and evaluating them against unit tests, training the critic network, and generating critic scores, along with the datasets used, including a compiled, permissively licensed subset of public GitHub code and the CodeSearchNet dataset.
It is written in Python, requires an included, modified version of the Hugging Face Transformers library, and is released under the BSD-3-Clause license.