RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning
A reinforcement-learning framework that trains an LLM to decide, turn by turn, when to reason, whether to retrieve from unstructured text or a knowledge graph, and when to answer, for adaptive hybrid retrieval-augmented generation.
RouteRAG trains a language model end to end with reinforcement learning to perform multi-turn, adaptive retrieval-augmented generation across both unstructured text and structured knowledge graphs, instead of relying on a fixed or handcrafted retrieval pipeline. At each step the trained policy decides whether to reason further, whether to retrieve from text or from the graph, and when to stop and produce a final answer, routing between the two retrieval modes based on the current reasoning state.
Training runs in two reinforcement-learning stages: the first optimizes task accuracy using an exact-match reward, and the second adds a retrieval-efficiency reward so the model learns to exploit hybrid evidence while avoiding unnecessary retrieval calls. The pipeline needs a served OpenIE model to construct the knowledge graph and a separate retrieval service, both run through vLLM, and is set up with a dedicated conda environment; it is written in Python.