ARC-AGI-1 Task Generator
A generator of fresh ARC-AGI-1-style reasoning tasks, matched to the public evaluation set's distribution, for testing frontier models on problems they have not seen before.
This repository generates new ARC-AGI-1-style tasks that resemble the public ARC-AGI-1 dataset, so that a model such as the accompanying BDH-CQ reasoning model can be evaluated on problems it is unlikely to have encountered during training.
Because the public ARC-AGI-1 benchmark cannot fully rule out prior familiarity with its tasks, arc-task-gen creates a private evaluation set with similar properties, letting a public-benchmark score be compared against performance on newly generated tasks. Generated tasks follow the standard ARC train/test JSON format and are compatible with existing ARC evaluation harnesses.