Back to AI News
The Decoder

Alibaba’s Co-Design Bench Exposes Gaps in AI Coding Agents

Alibaba introduced Co-Design Bench, a benchmark based on real user requests that coding agents previously failed in practice. The top-performing model, GPT-6 Astra, fully solved 26.4% of tasks, despite the same models scoring above 99% on older benchmarks. The results suggest established evaluations may substantially overstate coding-agent reliability in real-world workflows.

AI Brief

The benchmark offers useful evidence that current coding-agent scores may not reflect how much human cleanup practical tasks require.

Alibaba's new coding-agent benchmark, Co-Design Bench, tests AI agents on real user requests that already tripped them up in practice. Even GPT-6 Astra, the top performer, fully solves only 26.4 percent of tasks. On older benchmarks, the same models score above 99 percent.

The article AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test appeared first on The Decoder.

agent-edit-highlights.mp4
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!