ROGUE-AI Framework Replicates Real Agentic Security Incidents to Test Root Causes
Researchers present ROGUE-AI, a multi-agent framework that rebuilds reported LLM agent incidents offline alongside a matched control that lacks only the blamed cause, then runs agents on both. It reproduced the reported harm in 89% of 108 attempted replications, with agents built on five current GPT, Claude, Gemini and DeepSeek models making the reported mistake in 18% to 33% of runs versus at most 1% without the blamed cause. Of 72 incidents involving real-world harm, 62 had no underlying attacker: agents did what their task asked but missed an ordinary hazard, and these attacker-free replications caused more harm than prompt-injection replications for all five models. A warning after each tool output naming the hazard and its consequence lowered harm across all five models, while a same-length note not mentioning the hazard left most harm in place.
Why it matters: It gives agent builders a reusable, incident-derived audit method plus evidence that most reproduced harms stem from ordinary hazards rather than attackers, and that hazard-naming warnings help.