End-to-End Reinforcement Learning: Ensures the attainment of global objectives.
Textual Gradient Updates: Allows local nodes to flexibly adapt to dynamic tasks.
Node Skipping Mechanism: Significantly reduces computational overhead while maintaining accuracy.
Authors: Zheng Wang, Yuang Liu, Yangkai Ding
Affiliations: Huawei Poisson Lab, Wiener Research Center (NWRC)
Paper Link: https://arxiv.org/abs/2605.12943
Code Repository: https://github.com/openJiuwen-ai/agent-store/tree/main/community/mango
Core Methodology
- Flow Network Construction:
Actions from workflows are iteratively inserted into the network. To maintain transition integrity, adjacent operations are prevented from residing in the same node. New nodes are created based on vector similarity thresholds, with each node assigned a distinct Large Language Model (LLM), forming a multi-agent ecosystem. - Reinforcement Learning (RL) for Edge Optimization:
Given a constructed flow network, MANGO treats edge selection as a Markov Decision Process (MDP). State: Vector similarity derived from the current node's problem content and role description versus neighboring nodes.
Action: Selecting an outgoing edge (next agent).
Reward: A composite metric balancing process correctness and final task performance.
Policy: Optimized using the REINFORCE algorithm to maximize cumulative rewards.
- Textual Gradient for Node Optimization:
Each node's prompt (task content and role description) is updated based on both the final task outcome (global signal) and intermediate execution feedback (local signal). This ensures gradient signals remain strong even in long workflow chains, addressing the vanishing gradient problem. - Node Skipping Mechanism:
Recognizing that fully optimized nodes yield diminishing returns, MANGO introduces a "Skip-k" mechanism. By selectively skipping updates to well-optimized nodes and reusing proven intermediate steps, the framework reduces computational costs significantly. This creates a symbiotic relationship where parameter updates and path selection influence each other in an optimization loop.
Experimental Results
Effectiveness: MANGO achieved a 12.8% accuracy improvement over the best baseline (MaAS) on the MATH500 task and a 5.1% F1-score boost over AFlow on DROP.
Efficiency: Leveraging the Skip-3 mechanism, MANGO reduced API costs and runtime significantly. Compared to MaAS, it cut inference time by 47.4% and training time by 41.5% while maintaining the highest accuracy.
Comments & Questions (0)
No comments yet
Be the first to comment!