Evaluation of Multi-Agent Systems
A continuous-evaluation pipeline for multi-agent systems built with Google's Agent Development Kit and the A2A protocol on Cloud Run, deploying agents to shadow revisions and running automated evaluation suites to drive data-driven deployment decisions.
This project is a companion code repository to a codelab on moving from informal 'vibe checks' to data-driven agent evaluation. It implements a distributed multi-agent system where an orchestrator service delegates to a researcher agent, equipped with a Wikipedia search tool, a judge agent that scores the researcher's output for quality and relevance, and a content-builder agent that compiles verified information into a final report, with a web app letting a user trigger runs and watch progress; every agent runs as its own Cloud Run microservice communicating over the A2A protocol.
The continuous-evaluation piece safely deploys new agent versions to shadow Cloud Run revisions and runs automated evaluation suites against them using Vertex AI's Agent Platform GenAI Evaluation Service, which provides enterprise tooling for objective, data-driven assessment of generative AI models and agents, so a deployment decision can be based on measured evaluation results rather than a manual spot check.
It is built with Google's Agent Development Kit and is intended as a demonstration of agentic-engineering evaluation practices rather than a general-purpose evaluation library.