Back to AI News
arxiv

Securing Multi-Agent Systems by Design

Researchers propose response manipulation attacks (ReMA), which iteratively optimize malicious agent responses against multi-agent systems. They also introduce SecMAS, a secure aggregation rule that uses semantic similarity to select responses consistent with a sufficiently large subset of peers. Theoretical guarantees cover untargeted and targeted attacks, and evaluations show existing aggregation rules are vulnerable while SecMAS defends effectively with comparable utility.

AI Brief

Provides a new threat model and a theoretically grounded defense for securing multi-agent AI systems.

arXiv:2611.17883v1 Announce Type: cross Abstract: Multi-agent systems solve complex tasks by decomposing them into subtasks, delegating the subtasks to agents, and aggregating the agents' responses using an aggregation rule. Malicious agents can manipulate the final solution by providing malicious responses to their assigned subtasks. In this paper, we first propose response manipulation attacks (ReMA), which iteratively optimize malicious responses to maximize attack effectiveness. Specifically, we formulate untargeted and targeted attacks as optimization problems and develop an approach to approximately solve these problems by iteratively refining malicious responses based on feedback from a surrogate multi-agent system. To defend against such attacks, we further propose SecMAS, a secure aggregation rule that leverages the semantic similarities among agent responses to select the response that is most consistent with a sufficiently large subset of the other responses. We theoretically establish security guarantees for SecMAS against both untargeted and targeted attacks. Extensive evaluations demonstrate that existing aggregation rules are vulnerable to ReMA, whereas SecMAS effectively defends against all evaluated attacks, including adaptive variants of ReMA, while maintaining utility comparable to that of existing aggregation rules.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!