Evret
A lightweight Python framework for evaluating information-retrieval systems such as RAG and search pipelines, computing standard metrics like Hit Rate, Recall, NDCG, and MRR.
Visit website (opens in a new tab)kaivid-labs.github.io
Evret brings standard information-retrieval metrics to retrieval, recommendation, and RAG systems: Hit Rate, Recall, Precision, MRR, NDCG, ERR, RBP, and Average Precision, computed in a few lines of code. It supports judge-based relevance matching using token overlap, semantic similarity, or an LLM judge.
It includes adapters for vector databases such as Qdrant and for frameworks including LangChain and LlamaIndex, installed as optional extras. Development uses the uv toolchain and a pytest test suite.