Back to AI News
arxiv

CarryOn: Learned Context Management Boosts Long-Horizon LLM Agent Performance

Researchers introduce CarryOn, a lightweight context manager for long-horizon LLM agents that pairs compact execution state with fine-grained, source-linked evidence. A learned selector and adaptive retention policy keep relevant details available without extra LLM calls for context management, addressing the limits of summarization and pruning. Evaluated on AppWorld, OfficeBench, and Tau2 Telecom with three LLM actors, CarryOn achieves 75.7% macro-averaged task success, 6.7 percentage points above the strongest baseline, and the best scores on two proposed metrics, S_alpha and normalized cost per success (NCS).

AI Brief

Context management is a core bottleneck for reliable multi-step AI agents, and this paper delivers a concrete learned approach with benchmark-verified gains and new efficiency metrics.

arXiv:2611.17808v1 Announce Type: cross Abstract: Long-horizon LLM agents can coordinate tools and applications to complete complex, multi-step tasks. As their interaction histories grow, effective context management becomes essential for reliable and efficient execution. Existing methods reduce context through summarization, pruning, or history selection. However, shorter contexts do not necessarily support better decisions: summaries can omit exact details needed for later actions, while retaining entire interactions can carry substantial irrelevant information. We propose CarryOn, a lightweight context manager that pairs compact execution state with fine-grained, source-linked evidence. A learned selector and adaptive retention policy keep relevant evidence available without additional LLM calls for context management. We introduce two complementary evaluation metrics: $S_\alpha$ prioritizes task success while accounting for resource use, and normalized cost per success (NCS) measures the resources spent per successful task. Across AppWorld, OfficeBench, and Tau2 Telecom with three LLM Actors, CarryOn achieves 75.7% macro-averaged task success, exceeding the strongest baseline by 6.7 percentage points. It also achieves the best average scores on both metrics.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!