O²LaF Repurposes Video Diffusion Transformers for Latent Action Learning
Researchers introduce O²LaF, a method that initializes both inverse and forward dynamics models from the same pretrained video diffusion transformer. One model acts as a feed-forward transition encoder with learnable action queries, while the other is a causal, action-conditioned video generator. The framework is trained end-to-end through a compact bottleneck to learn transferable actions from unlabeled videos. The authors report state-of-the-art few-shot action probing and stronger zero-shot action transfer across visual contexts and embodiments, with competitive results also shown using V-JEPA2 priors.
It offers a concrete research approach for extracting transferable action representations from pretrained video models without labeled action data.
Comments & Questions (0)
No comments yet
Be the first to comment!