WATERFALL: Automated Reward Discovery for Partially Observable RL
Researchers introduce WATERFALL, a population-based workflow for automated reward discovery in reinforcement learning settings where task objectives and dynamics must be learned through interaction. The method generates diverse programmatic reward candidates, evaluates visual rollouts through a Swiss-system evolutionary tournament judged by vision-language models, and uses assessment histories to mutate, escalate, or prune candidates. Across fully observable and reveal-gated partially observable environments, WATERFALL reportedly outperformed sparse-reward RL, intrinsic-motivation, and foundation-model reward-synthesis baselines most clearly in the POMDP regime. Ablations indicate visual evidence, tournament-based selection, and iterative refinement are jointly important.
It presents a concrete research method for reducing manual reward engineering in RL without relying on privileged task information, which is useful for researchers working on autonomous reward design and POMDPs.
Comments & Questions (0)
No comments yet
Be the first to comment!