Inspect AI
A framework from the UK AI Security Institute for evaluating large language models, with built-in support for prompt engineering, tool use, multi-turn dialogue, and model-graded scoring.
Inspect AI is a framework for evaluating large language models, created by the UK AI Security Institute. It provides built-in components for prompt engineering, tool usage, multi-turn dialogue, and model-graded evaluations, and it ships with a collection of more than 200 pre-built evaluations that can be run against any supported model.
The framework can be extended by other Python packages to add new elicitation and scoring techniques. Documentation is published as a structured index for coding agents, with the full user guide concatenated as Markdown and individual pages available in Markdown by appending an extension to their URL. A web-based view for inspecting evaluation runs and results lives in a separate TypeScript frontend included as a git submodule.
It is written in Python and released under the MIT license.