# ARC-AGI-1 Task Generator

A generator of fresh ARC-AGI-1-style reasoning tasks, matched to the public evaluation set's distribution, for testing frontier models on problems they have not seen before.

This repository generates new ARC-AGI-1-style tasks that resemble the public ARC-AGI-1 dataset, so that a model such as the accompanying BDH-CQ reasoning model can be evaluated on problems it is unlikely to have encountered during training.

Because the public ARC-AGI-1 benchmark cannot fully rule out prior familiarity with its tasks, arc-task-gen creates a private evaluation set with similar properties, letting a public-benchmark score be compared against performance on newly generated tasks. Generated tasks follow the standard ARC train/test JSON format and are compatible with existing ARC evaluation harnesses.

Website: https://github.com/pathwaycom/arc-task-gen