Back to AI News
arxiv

Gym-Anything for Robotics uses coding agents to scale robotics simulation creation

The paper proposes Gym-Anything for Robotics, a framework in which coding agents build realistic robotics simulations, on the premise that simulation creation is itself a coding task that improves as coding agents improve. The authors select simulation domains based on U.S. GDP data and define criteria for good simulations and tasks, then apply an adversarial creation-audit loop. The resulting RoboWorld suite contains 25 simulations spanning 25 test and 175 training RL environments of varying difficulty. Frontier models reportedly average only 0.06 reward on RoboWorld test tasks, and the paper introduces in-context learning over repeated attempts, a privileged teacher-student setup, self-assisted learning, and code policies that generate demonstrations for training VLAs such as pi_0.5.

AI Brief

It offers a concrete pipeline, benchmark and released environments for scaling embodied-AI simulation without proportional expert labor, plus evidence that current frontier models still struggle on these tasks.

arXiv:2610.40565v1 Announce Type: cross Abstract: Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model's own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as $\pi_{0.5}$. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!