Back to AI News
arxiv

CARE-PO: Controllable Tool-Call Efficiency for LLM Agents

The paper diagnoses Qwen3.5 agent trajectories on GAIA and reports that most contain removable tool calls at both model sizes studied. It argues that existing efficiency rewards use policy-dependent anchors, can drift toward a collapsed no-call policy, and yield only one operating point per training run. CARE-PO instead prices calls against a reference budget fixed before training and anchored to task difficulty, injects a successful training trajectory into each GRPO rollout group, and uses an effort tag in the prompt to expose multiple efficiency-capability operating points from a single run. On GAIA, five of six operating points across three model sizes improve both success rate and call count, while results on four held-out multi-hop benchmarks are size-dependent.

AI Brief

It offers a concrete reinforcement-learning method for reducing LLM agent tool-call costs without discarding task success, with reported results across multiple model sizes.

arXiv:2611.12125v1 Announce Type: cross Abstract: External tool calls are a major running cost of an LLM agent, while its training objectives align almost exclusively with task completion. In a step-level diagnosis of Qwen3.5 trajectories at two model sizes on GAIA, a long-horizon tool-use benchmark, we find that most contain removable calls, at a similar rate at both sizes. Some methods add an efficiency reward to the objective to reduce the call count, but introduce problems of their own: the anchor they price calls against is computed from the policy's own rollouts (e.g., the group's shortest correct rollout) and so moves with the policy; training can drift toward a collapsed policy that issues no tool call at all, a degeneracy seen in published reports and in our 27B baseline; and a training run delivers a single operating point, so the user cannot trade efficiency against capability. We present CARE-PO (Controllable Anchored Relative Efficiency Policy Optimization) to mitigate these problems. The efficiency reward prices calls against a reference budget anchored to the task's own difficulty and fixed before training. Injecting a successful trajectory from the training data into each online GRPO rollout group and charging for evidence a trajectory never gathers ease the drift toward a collapsed call count. Shaping the cost curve by an effort tag in the prompt yields an efficiency and a capability operating point from a single training run. On GAIA, five of the six operating points across three model sizes improve on their base model in success rate and call count at once. On four held-out multi-hop benchmarks the result is size-dependent: at 27B and 4B success rate is held or raised at a lower call count, while at 9B both operating points trade success rate for fewer calls. The call count remains well above zero throughout training at all three sizes.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!