Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents
This arXiv paper compares three decision models and three general LLMs as actors and judges on ALFWorld, and also uses JEV as a teacher on ALFWorld and WebShop. As actors, all six models are pulled away from a subgoal that must come first by an easier one; as judges seeing the whole trajectory, the same models credit steps toward that subgoal, and asking whether each action helped separates useful from useless actions. As a teacher, JEV raises a Qwen2.5-1.5B policy trained with GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop. The authors conclude that a poor actor can still teach well, because judging an action after the episode ends reveals consequences that acting must anticipate, and they released code.
It offers a concrete finding that weak agent decision models can still serve as effective reward teachers, with measured gains and released code.
Comments & Questions (0)
No comments yet
Be the first to comment!