Research Editor selected
ScamWorld: Interactive Environment for Evaluating Web Agents Against Online Fraud
ScamWorld is a new interactive environment that reconstructs reported scam websites to test how web agents handle online fraud. In tests with six frontier LLMs, baseline deception rates ranged from 11% to 35%, with agents more likely to proceed with payments than with credential or wallet requests. The environment enables controlled, repeatable evaluation of agent behavior against real-world fraud scenarios.
Why it matters: Provides a concrete, novel testbed and empirical results on LLM-based web agents' vulnerability to online fraud, directly relevant to AI safety and agent evaluation.
AI safetyweb agent evaluationonline fraud