Vision is All You Need: V-RAG (Vision RAG) Demo
A demo implementation of Vision RAG, which embeds PDF pages directly as images with a vision-language model instead of chunking text, then retrieves and answers over the best-matching pages.
This is a demo of the Vision RAG (V-RAG) architecture, which embeds document pages as images using a vision-language model rather than extracting and chunking their text. PDF pages are converted to images, embedded with ColPali, and stored in a Qdrant vector database.
At query time, the query is embedded with the same vision-language model and used to search the vector database. The best-matching page images, along with the original query, are passed to a model that can read images, such as GPT-4o or GPT-4o-mini, which generates the final answer.
The demo runs as a Modal-served FastAPI application with a React frontend, using a Hugging Face account for the embedding model and an OpenAI API key for the answering model; the reference deployment uses an A10G GPU.