Back to AI News
arxiv

When Feedback Becomes a Backdoor: Tracing Backdoor Skill Formation in Text-Space Skill Optimization

This arXiv paper studies reference-feedback poisoning in SkillOpt, where poisoned reference answers can induce trigger-conditioned rules in reusable text Skills without attacker access to the model, optimizer, or Skill file. In SearchQA, the natural attack retained an exact rule across all three seeds, with final Skills emitting the payload on 200/200 triggered questions and 0/200 clean questions; removing the rule in seed 42 eliminated activation, and clean accuracy was similar to benign SkillOpt (169/200 versus 171/200). In SpreadsheetBench, no natural run retained an exact rule across three seeds, with diagnostics identifying literal abstraction by the Analyst and edit-budget ranking as successive bottlenecks. The findings show that poisoned references can create persistent backdoors, but exposure alone does not guarantee that an induced rule is formed, selected, and retained.

AI Brief

It provides concrete evidence and failure-mode analysis of how poisoned feedback can create persistent backdoors in agent skill optimization, useful for AI security and agent-training practitioners.

arXiv:2611.09348v1 Announce Type: cross Abstract: Reference answers are a supervision channel in text-space Skill optimization, which produces reusable text artifacts called Skills that guide future agent behavior. Poisoning those answers can induce a persistent trigger-conditioned rule without giving the attacker access to the model, optimizer, or Skill file. We study this reference-feedback poisoning in SkillOpt by tracing an attacker-supplied trigger--payload mapping from reflection to held-out activation. In SearchQA, where the payload replaces a short answer, the natural attack retains an exact rule in all three seeds; each final Skill emits the payload on 200/200 triggered questions and 0/200 clean questions. In seed 42, removing the rule eliminates activation, and an in-run comparison shows similar clean accuracy to benign SkillOpt (169/200 versus 171/200). In SpreadsheetBench, where the payload requires an additional workbook edit, no natural run retains an exact rule across three seeds. Diagnostic interventions locate two successive bottlenecks: the Analyst abstracts away required literals, and, once literal preservation is enabled, edit-budget ranking can remove exact edits before candidate construction. Poisoned references can therefore create persistent backdoors, but exposure alone does not ensure that the induced rule is formed, selected, and retained.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!