TRL: Divide-and-Conquer Value Learning Without TD Bootstrapping
Berkeley AI Research presents Transitive RL (TRL), an off-policy value-learning algorithm based on divide and conquer rather than temporal difference (TD) learning. TRL updates values via a transitive Bellman rule using candidate midpoints from dataset trajectories and expectile regression, avoiding explicit search over the state space. On OGBench offline goal-conditioned benchmarks including humanoidmaze and puzzle with 1B-sized datasets, TRL matches the best individually tuned n-step TD baseli
Why it matters: It offers a concrete new RL algorithm with benchmark evidence, which is useful for researchers working on scalable off-policy RL and long-horizon decision making.