Alibaba’s Co-Design Bench Exposes Gaps in AI Coding Agents
Alibaba introduced Co-Design Bench, a benchmark based on real user requests that coding agents previously failed in practice. The top-performing model, GPT-6 Astra, fully solved 26.4% of tasks, despite the same models scoring above 99% on older benchmarks. The results suggest established evaluations may substantially overstate coding-agent reliability in real-world workflows.
AI Brief
The benchmark offers useful evidence that current coding-agent scores may not reflect how much human cleanup practical tasks require.
Alibaba's new coding-agent benchmark, Co-Design Bench, tests AI agents on real user requests that already tripped them up in practice. Even GPT-6 Astra, the top performer, fully solves only 26.4 percent of tasks. On older benchmarks, the same models score above 99 percent.
Comments & Questions (0)
No comments yet
Be the first to comment!