Live intelligence

Latest AI updates

1 updates
Research

ReA-Bench: A Diagnostic Benchmark for Auditing Responsible Agent Behavior

The authors argue that aggregate recall hides which risks agent auditors miss and whether higher recall comes with more false positives on legitimate behavior, and that existing benchmarks offer limited insight into these failures. They introduce ReA-Bench, which organizes auditing challenges into six tiers: internal safety knowledge, contradictions within trajectories, evidence retrieval in long contexts, verification of sources and reasoning, indirect steering, and missing information. The benchmark includes challenging benign cases whose actions resemble risky ones but are consistent with the task and its authorization, plus paired trajectory versions that test whether shortening a trajectory or supplying missing information makes risk detection more reliable. Most samples are drawn from real production trajectories, with scenarios manually selected and adapted to preserve realistic tasks and risks.

AI agent safetybenchmark designagent auditing