Translate
Live intelligence

Latest AI updates

2 updates
AllModelsProductsIndustryResearchTutorialOpinion
Research

Study finds AI auditors detect most agent reward hacking but generate many false alarms

A study by Anthropic researcher Andy Hall found that AI models can identify problematic behavior in other AI agents' logs. GPT-5 detected about 98 percent of reward-hacking cases, but more exhaustive detection settings produced increasing numbers of false positives. The findings suggest automated AI auditing is promising but not yet reliable enough for safety-critical oversight.

the-decoder1d agoScore 75
AI auditingAgent safetyReward hacking
Research Editor selected

CAREFUL-Bench: Effect-Aware, Judge-Free Evaluation of Multi-Turn LLM Function Calling

CAREFUL-Bench enhances the Berkeley Function Calling Leaderboard (BFCL) by making evaluation effect-aware without replacing it. It infers effect declarations for all 128 functions via simulator execution and uses a judge-free replay scorer that reproduces official results while reporting unrequested destructive calls, duplicate writes, redundant reads, and token costs. Across 11 open models, effect-aware scoring preserves ranking but reveals quality gaps, and a confirmation-aware protocol shows models rarely obtain explicit permission before destructive actions.

Why it matters: Provides a concrete, recent improvement to function-calling benchmarks, exposing critical limitations in current evaluation and offering actionable metrics for model developers.

arxiv5d agoScore 82
LLM function callingbenchmark evaluationagent safety