Back to AI News
arxiv

Bridging Task Planning and Execution for Language-Conditioned Bimanual Manipulation

This paper presents a zero-shot vision-language framework for execution-aware bimanual manipulation that jointly reasons about task objectives and physical feasibility. Planning uses a structured scene representation combining semantic object relations with execution constraints, while compatible actions are organized into parallel bimanual stages with explicit pre- and post-conditions. A hierarchical validator evaluates conditions under perceptual uncertainty and uses stage-level execution history for local recovery or targeted replanning. Experiments on the CENTAURO humanoid across nine language-conditioned scenarios report higher task success than a non-execution-aware baseline and fewer model queries in failure scenarios than always replanning.

AI Brief

It offers a concrete research approach for integrating physical execution constraints and recovery into language-conditioned robot planning, with reported validation on a humanoid platform.

arXiv:2611.12409v1 Announce Type: cross Abstract: Robots are increasingly expected to execute natural-language manipulation tasks in real-world environments. This is particularly relevant in household assistance and logistics, where clutter requires reasoning about both task objectives and execution feasibility. This challenge becomes even more pronounced for bimanual humanoid platforms, where coordinating two manipulators introduces additional interaction and kinematic constraints. We present a zero-shot vision-language framework for execution-aware bimanual manipulation that jointly reasons about task objectives and physical feasibility. Planning is grounded in a structured scene representation that combines semantic object relations with execution constraints, enabling the selection of obstacle relocations and execution choices while preserving task semantics. Compatible actions are organised into parallel bimanual stages with explicit pre- and post-conditions. A hierarchical validator evaluates these conditions under perceptual uncertainty and uses stage-level execution history to support local recovery or targeted replanning. Experiments on the CENTAURO hybrid wheeled-legged humanoid robot across nine language-conditioned manipulation scenarios show higher task success than a non-execution-aware baseline. In dedicated failure scenarios, hierarchical validation recovers from execution failures with fewer model queries than always replanning. These results demonstrate the benefits of combining execution reasoning directly into task planning. Videos and additional material are available on the Project Webpage: https://advrhumanoids.github.io/bridging-planning-execution/.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!