AutoRefine
An RL post-training framework for retrieval-augmented reasoning that adds explicit knowledge-refinement steps between search calls, from a NeurIPS 2025 paper.
AutoRefine is the official implementation of a NeurIPS 2025 paper on a search-and-refine-during-think paradigm for retrieval-augmented reasoning. It introduces explicit knowledge-refinement steps between successive search calls so a model can iteratively filter, distill and organize retrieved evidence before generating an answer, alongside retrieval-specific rewards that guide searching behavior in addition to answer-correctness rewards.
Reproducing it requires a training and testing environment built on PyTorch, vLLM and a modified verl, plus a separate retrieval environment running a local FAISS-backed retrieval server. A released checkpoint is available on HuggingFace, and the authors report the pipeline can be reproduced end to end within an hour given a good connection.
A bundled Gradio demo lets a user enter questions and see the model's search-and-refine responses. It is written in Python.