Live intelligence

Latest AI updates

1 updates
Research

Accelerate Bench: Evaluating Agents for Application-Level Program Optimization

Researchers introduce Accelerate Bench, a benchmark of 32 CPU optimization tasks adapted from scientific applications and supercomputing competitions. Each task supplies a working program plus workload, correctness requirements, timing boundaries and permitted edits, with public testing for iteration and a separate verifier scoring the final submission. On version 1.1, 16 coding agents (model plus harness pairings) score between 25.5% and 67.4% Score@1, with most programs correct and speedup separating the leaders. The two leading agents reach at least 2x speedup on 22 or more of the 32 tasks, while the other fourteen hit 2x on only 8 to 17 tasks, leaving 125 of their 412 correct outcomes below 1.25x. Controlled replays show build and runtime settings alone can swing identical source code between 0.29x and 2.12x, highlighting the need for coordination across a full application.

coding agentsprogram optimizationbenchmarks