Study Localizes Causal Goal Representations in LLM Attention Heads
Researchers built a multi-sub-task coding setting to test whether LLMs encode goals as internal representations that guide action across different execution paths. They localized an encoding of the next sub-goal in the outputs of a sparse set of attention heads, which can also decode the model's upcoming actions in natural language. Causal interventions confirmed the encoding drives the next action and is plastic: transplanting it into another programming language triggers the same operation via different tokens. The authors propose a non-recurrent goal-seeking account in which the model repeatedly re-reads the goal from the task specification and compares it with visible progress, and they test this with targeted interventions.
Offers mechanistic evidence that LLM goal-directed behavior can be decoded and causally manipulated at specific attention heads, useful for interpretability and agent-reliability work.
Comments & Questions (0)
No comments yet
Be the first to comment!