WATERFALL: Automated Reward Discovery for Partially Observable RL
Researchers introduce WATERFALL, a population-based workflow for automated reward discovery in reinforcement learning settings where task objectives and dynamics must be learned through interaction. The method generates diverse programmatic reward candidates, evaluates visual rollouts through a Swiss-system evolutionary tournament judged by vision-language models, and uses assessment histories to mutate, escalate, or prune candidates. Across fully observable and reveal-gated partially observable environments, WATERFALL reportedly outperformed sparse-reward RL, intrinsic-motivation, and foundation-model reward-synthesis baselines most clearly in the POMDP regime. Ablations indicate visual evidence, tournament-based selection, and iterative refinement are jointly important.