arXiv:2610.40030v1 Announce Type: new
Abstract: Memory in reinforcement learning raises two problems. Attention finds the relevant item by scoring the whole store, which is costly for robots under latency and energy limits. When a retrieved memory pays off, credit for acquiring it must cross the whole sequence. We argue that both are governed by one quantity, the value of information, and propose to estimate it from archived rollouts with a learned world model. We obtain that, when read at the observation, it yields a provable lower bound on the optimal value of the acquiring action, which floors its TD target and attributes this floor to the observation that earned it. Read at the decision, it defines the optimal retrieval set. A pilot retrieval system, to be trained and improved with this signal, solves a memory task with reduced context.
arxiv
Value of Information Links Memory Retrieval and Credit Assignment in Reinforcement Learning
A new paper proposes using value of information, estimated from archived rollouts with a learned world model, to address both memory retrieval and credit assignment in reinforcement learning. The authors derive a lower bound on the value of an action that acquires an observation and use the same quantity to characterize optimal retrieval sets. A pilot retrieval system solves a memory task with reduced context, though the abstract does not report quantified latency or energy savings.
The work offers a unified theoretical approach to two memory bottlenecks in reinforcement learning, with preliminary evidence rather than demonstrated deployment benefits.
Comments & Questions (0)
No comments yet
Be the first to comment!