Sightline: Calibrated Data-Flow Summaries for LLM-Integrated Applications
The paper introduces Sightline, a data-flow analysis for LLM-integrated applications that summarizes which prompt placeholders an LLM output uses. It proves in Lean 4 that a placeholder flows to a transformer output, to first order, only through attention that reads it, and that faint reading or sensitivity cannot certify non-flow. Sightline reads a recorded call in one forward pass, learns which reads are uses from labels, and calibrates pruning to miss at most a chosen fraction alpha of true flows on average. On 2,103 unseen repository families and four models, it is reported as the most accurate summary against seven closest methods and a hidden-state probe, pruning 43.9-65.6% of spurious flows at alpha=1%.
It offers a concrete, calibrated method for tracing data flow through LLM calls, useful for security and information-flow control in LLM-integrated applications.
Comments & Questions (0)
No comments yet
Be the first to comment!