Back to AI News
arxiv

FrontierOR Challenge benchmarks LLM agents at operations-research algorithm design

The FrontierOR Challenge is a new benchmark and competition for agentic workflows that design, test, and improve optimization algorithms across many operations-research problems. Stage 1 releases 60 of 180 audited problems across six application categories and five formulation types in weekly rounds of 15, while Stage 2 adds new problems and unseen instances. Two tracks are ranked separately: a network-isolated sandbox using open-source dependencies and a self-hosted track allowing any software within a timed window. Organizers report that in one-shot evaluation the strongest model matched Gurobi within 1% objective and no slower on only 39% of large instances.

AI Brief

It provides a concrete, audited benchmark and competition format for measuring whether LLM agents can produce scalable operations-research algorithms.

arXiv:2611.26171v1 Announce Type: new Abstract: Operations research (OR) relies on algorithms that experts tailor to each problem: a new routing, scheduling or network design problem can take weeks of specialist work before it is solved well at scale. Large language model (LLM) agents now write, test and improve code, but whether they can design algorithms that scale to realistic instances is open: in the FrontierOR benchmark's one-shot evaluation, the strongest model matches Gurobi, with an objective within 1% of Gurobi's and no slower, on only 39% of the large instances. The FrontierOR Challenge asks teams to build agentic workflows that design, test, and improve algorithms for many optimization problems at once. Stage 1 releases 60 of the benchmark's 180 audited problems, spanning six application categories and five formulation types, in weekly rounds of 15; Stage 2 introduces new problems outside the benchmark, with unseen instances. Two tracks are ranked separately: in the Main track we run the submitted algorithms in a network-isolated sandbox with open-source dependencies only; in the Self-hosted track teams solve the instances on their own machines, with any software, within a timed window. Every solution is checked for feasibility, and a feasible solution scores the ratio of its objective to a frozen known reference, averaged over hidden instances.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!