Latest AI updates

Terminal-Bench 2.1: Continuous Benchmark Maintenance Reveals Stale Evaluations

The paper introduces the concept of benchmark maintenance debt, where benchmarks become stale as models improve and infrastructure changes. The authors propose a continuous-validation workflow and apply it to Terminal-Bench 2.0, finding issues in roughly 30% of the benchmark and releasing repairs as Terminal-Bench 2.1. These repairs change evaluation results, with accuracy increasing by over 10 percentage points and leaderboard positions shifting by up to three ranks, while overall ranking remains highly correlated.