Opik
An open-source platform for tracing, evaluating and monitoring LLM applications, RAG systems and agentic workflows, covering the full lifecycle from development traces to production dashboards.
Opik is an open-source LLM observability and evaluation platform for AI agent tracing, LLM evaluation, prompt management and production monitoring, covering the full lifecycle of an LLM application from its first development trace through production monitoring.
It traces LLM calls and multi-step agent activity, including tool calls, with feedback annotations on traces and spans, and a prompt playground for experimenting with prompts and models. Its evaluation tools run datasets and experiments with LLM-as-a-judge metrics for tasks such as hallucination detection, moderation and RAG assessment, and a PyTest integration runs these evaluations in CI/CD. Guardrail features help implement safe AI practices, and an Agent Optimizer SDK helps improve prompts and agents. It integrates natively with a range of frameworks, including recent additions such as Google ADK, Autogen and Flowise AI.
It is free to self-host in full, is Apache-2.0 licensed, and is built by Comet. It is written in Python.