Code-as-World
A research project that has an AI agent discover executable code representations of the physical world from video, so a vision-language model can verify and reuse explicit states, dynamics and mechanisms instead of only matching pixels.
The project's premise is that a pixel-level observation is evidence of the physical world, not its structure: it records how the world looked at one moment from one viewpoint, without saying what exists, how it is organized, or what governs how it evolves. Code-as-World turns raw video observations into reusable, executable representations, explicit states, dynamics and mechanisms, discovered agentically through iterative simulation and verification, rather than only pattern-matching pixels.
The released code supports local inference and evaluation on QuantiPhy, a physical-reasoning benchmark, for two vision-language checkpoints, Code-as-World-VL-4B and Code-as-World-VL-9B, plus a video-driven abstraction example; the authors report state-of-the-art results on quantitative physical reasoning from this physical supervision.
Running the released evaluation needs Python 3.10 or 3.11 on a CUDA host, the model weights downloaded from Hugging Face, and a separate clone of the QuantiPhy repository and its validation-video dataset.