AI evals

Reviewed resources in AI evals. 269 resources listed.

Opik
comet.com
An open-source platform for tracing, evaluating and monitoring LLM applications, RAG systems and agentic workflows, covering the full lifecycle from development traces to production dashboards.
- AI evals
Promptfoo
promptfoo.dev
A CLI and library for evaluating and red-teaming LLM-based applications, comparing model providers side by side and scanning for security vulnerabilities from the command line or CI/CD.
- AI evals
- Security
MLflow
mlflow.org
An open source platform for debugging, evaluating, monitoring and optimizing production AI applications built with agents, LLMs and machine learning models, while tracking cost and access.
- AI evals
- Monitoring
Langfuse
langfuse.com
An open-source LLM engineering platform for tracing, evaluating and debugging AI applications, with prompt version management and an interactive playground for iterating on prompts.
- AI evals
Prompt Optimizer
prompt.always200.com
A tool for iteratively optimizing, testing and comparing AI prompts across multiple models, available as a web app, desktop app, Chrome extension or Docker deployment.
- AI evals

Opik