CoDA-Bench

Given a natural-language question and a Linux sandbox containing hundreds of data files drawn from Kaggle datasets, an agent must discover which files are relevant among many semantically similar candidates, navigate complex directory hierarchies, integrate information across multiple heterogeneous sources, and generate correct analysis code, rather than being handed the right data directly as most existing benchmarks do.

The full benchmark contains 1,009 tasks across 31 communities drawn from 199 Kaggle datasets, averaging about 980 files per task and roughly 43 gigabytes of data in total, with a 119-task hard subset for faster iteration. Evaluation runs inside Docker, and a setup script downloads and extracts the benchmark data and all community archives in one step before an evaluation script runs a chosen model against a chosen set of task instances.