MultiHop-RAG
A question-answering dataset and evaluation code for testing retrieval and reasoning across multiple documents in retrieval-augmented generation pipelines, with 2,556 queries whose evidence spans two to four documents each.
MultiHop-RAG's queries involve document metadata as well as content, reflecting the kind of complex, cross-document scenarios common in real-world RAG applications, rather than single-document lookup. It is accompanied by sample scripts for retrieval (simple_retrieval.py, with an optional reranking pass) and for question answering with a Llama model (qa_llama.py), plus separate evaluation scripts for retrieval and QA.
Part of the code used to construct the dataset is released for research purposes, under the pipeline/ directory, though the authors note its structure is not yet fully organized. A Hugging Face dataloader is provided alongside the GitHub repository, and the dataset accompanies a paper accepted at COLM 2024.
The dataset itself is released under the ODC-BY license.