Live intelligence

Latest AI updates

1 updates
Research

ClosureBench: Benchmarking LLM Reasoning Under Closure Assumptions

Researchers introduced ClosureBench, a benchmark that tests whether LLM agents follow stated rules for handling missing information. Each base scenario is evaluated under open-world, closed-world, and local closed-world assumptions, with only the treatment of missing information changing between variants. The benchmark contains 960 instances and agent extensions for action choice, source scope, and updates; a scenario counts as correct only if all closure variants are answered correctly. Overall accuracy ranged from 71.18% to 99.86%, but core switch accuracy on scenarios whose gold label changes ranged from 18.23% to 99.31%, with large discrepancies under LCWA.

LLM evaluationreasoning under uncertaintyknowledge representation