WebArena-Pro: A Reproducible Multimodal Benchmark for Web Agents
WebArena-Pro is a new benchmark for web agents comprising 300 tasks across 20 self-hosted web applications in six domain categories. It extends evaluation beyond text and vision by treating audio and video as core observations, with dedicated actions for extracting information from each. The abstract reports top performance of 62.3% from Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5, while the strongest GenericAgent configurations reached 30.3%. The benchmark runs tasks in isolation to support reproducible parallel evaluation and aims to combine live-benchmark coverage with human-validated reproducibility.