ECOS: Environment-Calibrated On-Policy Self-Distillation for Language Agents
The paper introduces ECOS, a method that uses environmental outcomes to calibrate how much a language agent trusts its own Teacher guidance during on-policy self-distillation. Command-level advantages scale Teacher preferences and determine which self-written reflections and skills are reinforced. Without skills at inference, 3B and 7B agents reach 90.5% and 93.3% macro success on seen ALFWorld tasks and 75.5% and 80.5% success on WebShop. On unseen ALFWorld tasks, they outperform GRPO under the same protocol by 11.0 and 7.6 percentage points, and an ablation shows supervising only executable commands raises 3B WebShop success from 52.3% to 75.0%.
It presents a concrete training method and comparative results for improving language agents with environment-calibrated self-distillation.
Comments & Questions (0)
No comments yet
Be the first to comment!