Back to AI News
arxiv

Scale-Induced Semantic Shift Enables Visual Prompt Injection in VLMs

Researchers describe Scale-Induced Semantic Shift (SISS), a security failure mode in which image resizing or compression changes machine-readable semantics and makes hidden textual instructions recoverable after aggressive downsampling. Their closed-loop black-box framework reaches a 58.75% average attack success rate on a fixed 300-case benchmark across four closed-source vision-language models, though the effect is bounded to aggressive compression. In a downstream web-agent study, hidden-instruction perception increases but absolute behavioral compromise remains about 4%. The proposed ScaleGuard consistency detector reduces attack success, while generic input transformations provide little protection.

AI Brief

Useful for AI security teams because it documents a reproducible, bounded preprocessing vulnerability in vision-language models and evaluates both attack and defense.

arXiv:2611.16744v1 Announce Type: new Abstract: Visual content in multimedia systems may be resized, compressed, or re-rendered before it is interpreted by a vision-language model. We study a security failure mode in which these transformations change the machine-readable semantics of an image: a textual instruction is not recovered at a $512\times512$ presentation resolution under our evaluation protocol but becomes recoverable after aggressive downsampling. We formulate this behavior as Scale-Induced Semantic Shift (SISS) and develop a closed-loop black-box framework that combines dual-scale verification, selective self-hardening, shallow-layer low-rank adaptation, and pixel-space noise regularization. On a fixed 300-case benchmark, the third SISS refinement round obtains a $58.75\%$ average attack success rate across four closed-source vision-language models, compared with $41.08\%$ for the strongest generation baseline. A controlled sweep bounds the effect to aggressive compression: average success decreases to $17.68\%$, $6.85\%$, and $5.60\%$ at $128\times128$, $224\times224$, and $336\times336$, respectively. In a downstream web-agent study, the generated images increase hidden-instruction perception, but absolute behavioral compromise remains about $4\%$. Generic input transformations provide little protection in a separate defense protocol, whereas the proposed ScaleGuard consistency detector reduces non-adaptive success from $61.4\%$ to $24.3\%$ and one-round defense-aware success from $42.5\%$ to $15.0\%$, with no observed rejection among matched clean controls. The results characterize a bounded but reproducible semantic inconsistency in multimedia preprocessing and motivate scale-consistent visual-language inference.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!