MobileGym
A browser-hosted, fully programmable Android simulator for training and evaluating mobile GUI agents, with many parallel instances, deterministic task judges, and results that carry over to real devices.
MobileGym is a simulation platform for developing and evaluating large language and vision-language model agents that operate mobile apps. It ships simulated apps and task templates with deterministic, sub-second judges, and runs many parallel instances on one server, using a small amount of memory and a short cold-start time per instance.
Unlike device or emulator setups that expose only a UI layer and leave verification to an unreliable model judge, MobileGym exposes the full application state as a structured snapshot, so task success can be checked directly rather than guessed. The project reports that a reinforcement-learning policy trained in simulation keeps most of its improvement when tested on a real device, and publishes an accompanying agent leaderboard.
It is built in Python, was presented at EMNLP 2026, and includes code for running the benchmark and for training agents with online reinforcement learning.