Back to AI News
arxiv

Experience Rot: Experienced AI Agents Fail After Silent Tool Changes

A new arXiv paper reports that agents using tools defined via the Model Context Protocol can fail after a silent interface change, even though agents new to the task mostly succeed. The experienced agents tend to repeat earlier tool calls instead of checking the current interface, even when the changed definition is visible, and then report success. The failure appears across every memory form tested and grows with the amount of experience; announcing the update barely helps, and documentation pointers fail when the documentation does not spell out the answer.

AI Brief

It identifies a concrete reliability risk for memory-enabled tool-using agents and shows that common mitigation advice may not prevent failures after tool interfaces change.

arXiv:2611.20922v1 Announce Type: cross Abstract: Agents are increasingly built to learn from experience: they keep conversation histories, memory files, retrieved notes and skills, and reuse what worked before. We show that this experience turns against them when their tools change silently. We test a range of models with tools defined as in the Model Context Protocol (MCP). After a silent change, experienced agents fail on most tasks, while agents new to the task mostly succeed. We call this failure Experience Rot. Experienced agents copy their earlier call instead of checking the current interface, even when the tool definition shows the change. Then they report success. When the documentation is incomplete, experience is what makes agents succeed. After a change, the same experience leaves them worse off than no experience at all. The failure appears in every form of memory we test and grows with the amount of experience. Simple remedies fall short: announcing the update barely helps, and pointing the agent to the documentation fails whenever the documentation does not spell out the answer.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!