Back to AI News
arxiv

Shibboleth Traps Fingerprint LLM Models and Harnesses Behind Agentic Attacks

This arXiv paper studies fingerprinting agentic cyberattacks by inferring the attacker's LLM version and harness from telemetry. The authors introduce "shibboleth traps" — modular honeypot services that condition access to resources on informative interactions, eliciting model declarations and characteristic writing without needing a successful prompt injection. For harness fingerprinting, they use indirect probes that reveal secret masking in tool output, command time limits and tool-call scheduling from effects visible in later requests. Combined declarative and behavioral inference distinguished among 15 candidate models at an average 90% top-1 accuracy, and fixed probe rules identified the correct harness among four with 92% accuracy; agents instructed to conceal identity still left behavioral evidence.

AI Brief

It offers defenders a concrete attribution technique for identifying which models and agent harnesses drive agentic attacks, even when agents are told to hide their identity.

arXiv:2611.12386v1 Announce Type: cross Abstract: We study the fingerprinting of agentic cyberattacks. Our objective is to reliably infer information about the attacker's configuration, such as LLM version and harness, from telemetry. To achieve this, we introduce a technique we call shibboleth traps: modular honeypot services designed to elicit discriminative behaviors from agentic attackers. The traps make informative interactions a condition for obtaining resources the agent seeks, eliciting model declarations and characteristic writing without requiring a successful prompt injection. To achieve harness fingerprinting, we introduce general mechanisms to indirectly probe peculiar execution behaviors of harnesses. These probes reveal how a harness masks secrets in tool output, enforces command time limits, and schedules tool calls, using only the effects visible in subsequent requests. We show that combined declarative and behavioral inference distinguishes among 15 candidate models with an average 90% top-1 accuracy, with behavioral classifiers evaluated on held-out harnesses or task families. In a separate experiment across four harnesses, fixed probe rules identify the correct harness with 92% accuracy. When instructed to conceal their identity, agents still leave useful behavioral evidence even as their declarations become uninformative.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!