Back to AI News
arxiv

Study: Exposing Success Criteria to Embodied Agents May Cause Overfitting, Not Better Task Learning

A new arXiv paper investigates the practice of exposing benchmark success criteria to self-improving embodied agents, a method called Criteria Exposure (CE). Controlled experiments show that agents with CE achieve higher scores under the given criteria, but the advantage disappears when criteria are slightly altered without changing instructions. The skills developed under CE transfer poorly across tasks, suggesting overfitting to the verifier rather than learning the task. The authors propose instruction-faithful criteria to better reflect actual task performance.

AI Brief

This research provides critical insights into evaluation methodology for embodied AI agents, highlighting a potential pitfall in benchmark design that affects the validity of reported results.

arXiv:2611.08318v1 Announce Type: new Abstract: Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!