PIPETTE Turns Red-Team Discoveries Into Harder-to-Inject LLM Agents
Researchers introduce PIPETTE, a purple-teaming framework for indirect prompt injection in LLM agents. The paper shows that two state-of-the-art RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse, so high attack success rates during red-teaming do not translate into robustness gains from adversarial training. PIPETTE adds a dictionary learning-based reward to broaden attack strategy coverage and an on-policy adversarial training pipeline tailored to prompt injection defense. On AgentDojo and ASB, defenders trained with PIPETTE reach near-zero attack success rates on static attacks and withstand strong adaptive attacks while keeping benign task utility close to an undefended baseline.
It offers practitioners a concrete method for converting prompt injection red-teaming output into measurable agent robustness rather than redundant attacks.
Comments & Questions (0)
No comments yet
Be the first to comment!