Back to AI News
arxiv

Top-k Retrieval Leaves Memory Value Undetermined; LEDGER Estimates Determined Values

This paper analyzes memory-augmented agents and shows that top-k retrieval leaves memory values undetermined, even with many logged episodes. The authors introduce LEDGER, which estimates determined Shapley values with confidence intervals and reports others as undetermined, improving the fraction of determined memories on benchmarks.

AI Brief

Provides a novel theoretical result and a practical method for memory valuation, directly useful for AI memory system design.

arXiv:2611.03289v1 Announce Type: new Abstract: A memory-augmented agent must decide which memories to keep, merge and retrieve, and its only evidence of their value is its retrieval log: the memories it retrieved in each episode and whether it succeeded. Existing methods assign every memory a value from this log, even when the log cannot separate one memory's contribution from another's. We treat the log as an experiment that nobody designed and ask which memory values it determines. Two counts answer this before any outcome is observed: how often each memory was retrieved, and how often each pair was retrieved together. From these counts, we prove that top-$k$ retrieval leaves every memory value undetermined, however many episodes are logged, and that a question with its own memories, asked once, determines at most one of their values. On fourteen standard memory benchmarks, no memory value is determined in any of 55 configurations. Both causes are choices about retrieval. We introduce LEDGER, which estimates each determined Shapley value in closed form with a confidence interval and reports every other value as undetermined. It merges memories with identical histories without losing a determined value, and it retrieves so that the next log determines more. On pooled HotpotQA, LEDGER raises the fraction of memories determined from 0.001 to 0.797, at a cost of 2.6 accuracy points on HotpotQA and with a gain of 2.7 on LoCoMo.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!