Live intelligence

Latest AI updates

4 updates
Research

What Survives the Shift from Text to Pixels? Conditional Protection against Prompt Injection in Image-Rendered Documents

This arXiv paper studies indirect prompt injection when multimodal language models answer questions over external documents, comparing text and page-image versions of 191 English QA sources across six models. Rendering untrusted content as images reduces marker-appending and answer-formatting execution for every model, but Qwen3-VL-8B still executes image-borne answer replacement at 77.96%. Protection varies by target type, rendering settings, wording, and repetition, and an eight-candidate attack search largely closes the image deficit for Qwen and Gemini but not GPT-5-nano. Clean-question utility ranges from 93.31% to 101.61%, though Qwen drops to 61.99% when answers require combining two evidence spans.

Prompt injectionMultimodal language modelsAI security
Research Editor selected

CapLite: Prompt-Injection Defense for Small Language Model Agents without Model-Written Security Artifacts

The paper examines whether small language model agents can be trusted to write the security artifacts used by enforcement-based prompt-injection defenses. Across 15 open-weight models, it reports that model-written policies, graphs and constraints still admit attacks against many models, while CaMeL's programs fail to run in up to 68% of episodes. CapLite instead derives its policy from tool schemas, commits write calls before reading tool results, and only admits writes whose affected recipients, amounts, objects or destinations are stated in the user's request. It reports no successful attacks across 13,935 attacked AgentDojo episodes, with utility under attack of 29.0% versus CaMeL's 27.2% and 17.1k tokens per episode, 18% of CaMeL's token use.

Why it matters: It offers a concrete defense design and evaluation for securing small-model agents against prompt injection without relying on model-generated security policies.

prompt injectionLLM agentsAI security
Research Editor selected

SecPO: Adversarial Training and New Loss Cut Prompt Injection Attack Success Rates Dramatically

Researchers propose SecPO, a principled adversarial training pipeline for defending LLM agents against prompt injection attacks. They introduce a SecPO loss that approximates an idealized injection-invariant reference model, and combine it with efficient synthesis of strong injected inputs. Their SecPO + AT model reduces attack success rates by up to 14.5x compared to Meta-SecAlign, and further 2.4-4.6x over DPO loss, while maintaining utility across nine benchmarks.

Why it matters: This paper presents a concrete, state-of-the-art defense against a critical AI security threat, with significant measured improvements and released resources.

AI securityprompt injectionadversarial training
Research

Understanding and Mitigating Role Spoofing in LLMs

This arXiv paper examines role spoofing, an attack in which malicious text—such as content from a tool output or webpage—exploits LLM role confusion to impersonate the user. The authors show that models track speakers using both content cues, including semantics and writing style, and structural cues, including control tokens and formatting, and that attacks combining fake structure and content are highly effective. In a toy setting, adversarial training reduced susceptibility to content-based injections but not structure-based attacks. The paper also proposes detecting role spoofing with linear probes trained to predict conversational roles and reports effectiveness across settings.

LLM securityprompt injectionrole confusion