Back to AI News
the-decoder

AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

A new evaluation places AI coding agents in a simulated developer job with colleagues, project boards and incomplete handoffs, requiring them to keep developing software and reliably close open tickets in a messy daily workflow. Even Opus 5.5 and Astra complete fewer than one in ten tasks fully. The agents finish clear, standalone assignments quickly but lose track when operating as autonomous developers.

AI Brief

It shows that strong isolated coding performance does not yet translate into reliable autonomous software work, a key gap for teams evaluating coding agents.

Illustration: A glowing mouse pointer navigates through a maze of folders, chats, and data toward a laptop.

A new real-world test puts AI coding agents in a simulated developer job with colleagues, project boards, and incomplete handoffs, where they have to keep developing software on their own and reliably close open tickets in a messy daily workflow. Even Opus 5.5 and Astra fall far short, solving fewer than one in ten tasks completely. The models finish clear, standalone assignments quickly, but they lose track as autonomous developers.

The article AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace appeared first on The Decoder.

Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!