# VisRAG

A vision-language-model-based retrieval-augmented generation pipeline that embeds documents as images instead of parsed text, and its newer EVisRAG variant that reasons over per-image evidence for multi-image questions.

VisRAG is a vision-language model (VLM) based RAG pipeline that embeds a document directly as an image with a VLM instead of first parsing it into text, then retrieves those image embeddings to enhance a VLM's generation. Compared to text-based RAG, this avoids the information lost when parsing a document.

EVisRAG, described as VisRAG 2.0, is an evidence-guided extension for multi-image questions: it first has a model linguistically observe retrieved images to collect per-image evidence, then reasons over that evidence to answer, training with Reward-Scoped GRPO for token-level rewards that jointly optimize visual perception and reasoning.

The project releases benchmarks, training and test data, and trained models such as EVisRAG built on Qwen2.5-VL, and reproduces within the UltraRAG v2 project.

Website: https://github.com/OpenBMB/VisRAG