An Augmented Lagrangian Approach to Lifted Bellman Linear Programming
This paper introduces AL-LBLP, an augmented Lagrangian method for solving Lifted Bellman Linear Programming in offline reinforcement learning. Instead of fixed hinge-penalty coefficients, it learns multipliers via dual ascent with a single penalty parameter and shows that in the convex case fixed points are KKT pairs of LBLP. The critic can drive a Gaussian actor, a flow actor, or best-of-N sampling, and for the Gaussian actor the authors propose an advantage-normalized DDPG+BC objective. Using one Q and V and no target network, AL-LBLP with best-of-N reports 60.1% average success across 50 state-based OGBench tasks, 81.5% on humanoidmaze-giant, 79.8% on antmaze-giant, and no collapse up to batch size 8192.
It offers a concrete offline-RL algorithm with a cleaner optimization formulation and strong reported benchmark results, useful for researchers working on value-based reinforcement learning.
Comments & Questions (0)
No comments yet
Be the first to comment!