AEGIS: Automated Error Generation and Attribution for Multi-Agent Systems
A large-scale dataset and benchmark for detecting errors in multi-agent systems, with systematically generated failure scenarios and verifiable ground-truth labels across several agent frameworks.
AEGIS provides a dataset and benchmark for evaluating methods that detect errors in multi-agent systems. Its error-injection factory generates failures through prompt injection into agent inputs, direct corruption of agent responses, and context-aware attacks that adapt to the task domain and the target agent's role.
Standardized wrappers apply these injected errors across several multi-agent frameworks, including DyLAN, AgentVerse, LLM Debate, MacNet and Magnetic-One, and failures are labeled against a 14-mode taxonomy adapted from the MAST classification, covering specification issues, inter-agent misalignment and task-verification failures. The accompanying evaluation scripts score error-detection methods against both the AEGIS benchmark and a separate Who&When benchmark, and the work was accepted at ICLR 2026.