MultiWorld: Scalable Multi-Agent Multi-View Video World Models
A research implementation of a unified world-modeling framework that generates consistent multi-camera video while controlling multiple agents at once, scaling across different numbers of agents and camera views.
MultiWorld is the official implementation accompanying a research paper on multi-agent, multi-view video world models. Its Multi-Agent Condition Module uses agent identity embeddings and adaptive action weighting to control multiple agents accurately, while its Global State Encoder uses a frozen VGGT backbone to extract implicit 3D environmental information from partial observations, improving consistency across camera views; the model supports autoregressive inference to generate video beyond the length it was trained on.
The repository provides training code and long-term inference code, along with two released datasets used in the paper, a two-player video game dataset and a robotics dataset, plus pretrained checkpoints at two resolutions. It requires a Conda environment with PyTorch and CUDA 12.8, and models and data are downloaded from ModelScope or Hugging Face.