3D GameDev Arena benchmarks coding agents on long-horizon browser game development
Researchers introduce 3D GameDev Arena, a benchmark of 108 from-scratch 3D browser game-development tasks spanning 14 genres, where an automated plan-implement-test-repair pipeline builds each game from a text spec and UI/UX reference images. Runs average roughly 1,000 LLM calls, with the longest using 14,407 calls, 2.13B tokens, and 608 assets. Evaluation combines code-level functional acceptance, agent-based QA and playtesting, and blind pairwise human preference in a public arena. Preliminary findings indicate long-horizon 3D game development remains far from solved and that code-level correctness alone does not determine player preference.
It offers a concrete, public benchmark for evaluating coding agents on extreme long-horizon, multi-stage tasks with human preference signals, useful for tracking agent capability beyond repository-level issue fixing.
Comments & Questions (0)
No comments yet
Be the first to comment!