Understanding and Mitigating Role Spoofing in LLMs
This arXiv paper examines role spoofing, an attack in which malicious text—such as content from a tool output or webpage—exploits LLM role confusion to impersonate the user. The authors show that models track speakers us...