Live intelligence

Latest AI updates

1 updates
Research Editor selected

Embodied First Exam benchmark tests coding agents on robot control

The paper introduces Embodied First Exam, a unified agent-native benchmark for coding agents that control robots. It covers 1,156 tasks across 35 suites and 47 robots, including new suites for spatial puzzles, dexterous manipulation, multi-robot interaction, dynamic whole-body control, and sim-to-sim engineering. Agents can choose direct control, code-to-control, or policy training, with Standard, Realtime, and Privileged evaluation settings. On 456 core tasks, the strongest of eight agents, GPT-6 Astra in Codex, reaches a 34% success rate in the Standard Setting.

Why it matters: It offers a concrete, broad benchmark and baseline results for assessing whether coding agents can reliably operate robots under physical-world constraints.

Embodied First Examcoding agentsrobotics benchmarks