# XRAG

A benchmarking framework for evaluating the foundational components of retrieval-augmented generation systems, comparing text splitters, retrievers, and orchestrators against multiple evaluation metrics.

XRAG is a benchmarking framework for dissecting and evaluating the core modules of retrieval-augmented generation (RAG) systems. It measures how different configurations and components affect overall RAG performance, using LLM-based evaluation, deep evaluation, and traditional metrics to judge retrieval quality, response faithfulness, and answer correctness.

It includes multiple text splitters, such as semantic, sentence, and sentence-window node parsers, and multiple orchestrators, including self-RAG, adaptive-RAG, and similarity-guided RAG. It can also generate its own question-answer pairs from a folder of documents, supports Ollama models, and can run as a backend service through its API, with a web interface for interactive use.

It is written in Python, licensed under Apache-2.0, and accompanies a paper accepted at ICDE 2026.

Website: https://docailab.github.io/XRAG/