decision-lab
A harness for agentic data science that runs an analysis multiple independent ways and checks whether the results converge, instead of trusting a single AI-generated model and conclusion.
decision-lab addresses a specific failure mode: a coding agent given an open-ended analysis task picks one path through the analytical space and commits to it, and if that path is wrong the result is a polished-looking report with bad conclusions. decision-lab instead runs an analysis several ways, using different models or assumptions, through parallel subagents exploring different data preparation or model structures, and checks whether they converge; convergent results are reported as trustworthy, and divergent results are flagged along with what experiments would resolve the uncertainty.
Everything an agent needs, prompts, domain skills, tools and a locked environment, is packaged into a decision-pack pointed at a dataset, and sessions run by default in a Docker container with pinned dependencies so the agent always codes against the API version its skills were tuned for. On an adversarial marketing-mix-modeling dataset with no valid inference possible, the authors report that plain Claude Code fit a model and recommended a budget change anyway, while their decision-lab agent tried 11 approaches, found none converged, and recommended further data collection instead. Python, Apache-2.0 licensed, with compute-heavy tasks supported on Modal.