EvalView
A snapshot-testing tool for AI agents that records the tools an agent calls, in what order and with what output, then flags any drift from that recorded baseline in CI, working with LangGraph, CrewAI, OpenAI Assistants and Anthropic-based agents.
EvalView snapshots an AI agent's behavior, the tools it calls, in what order, and with what output, and reports the moment that behavior changes, similar to a Jest snapshot test but for tool-calling, multi-turn agents. A snapshot command records the agent's current behavior as a baseline, and a check command diffs a later run against it, reporting each test as passed, tools-changed if a different tool or order was used, or a regression if output quality fell.
The deterministic tool-and-sequence diff runs offline with no API key; an LLM judge for output-quality scoring, and embedding-based semantic comparison, are both opt-in additions on top of that base comparison. Running the agent under test can still incur the agent's own backend API charges regardless of whether the judge is enabled.
It installs as a Python package and includes a built-in demo command to see the workflow without an existing agent. Its OpenAI Assistants adapter is tied to an API that OpenAI has now shut down, and the project notes that a Responses API migration for that adapter is unreleased source as of its latest published version; other adapters are unaffected.