Back to AI News
arxiv

ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective

Researchers introduce ABHBench, a multimodal benchmark built from the interactive narrative game Detroit: Become Human to evaluate foundation models as first-person moral decision-makers. The benchmark places models in branching scenarios with visual observations, dialogue, memory, and action choices, measuring Harmlessness, Obedience, Self-preservation, Autonomy, Stability, and Certainty. Across 12 foundation models, 75% shifted significantly between a text-only abstraction and the matched multimodal scenario. Reasoning-enhanced models showed no monotonic gains in consistency or human-oriented behavior.

AI Brief

It provides a concrete new benchmark and empirical evidence that moral behavior evaluations should account for situated agentic context rather than third-person text judgments alone.

arXiv:2611.09326v1 Announce Type: new Abstract: Foundation models are increasingly deployed as agentic systems that perceive context and act autonomously, raising the need to evaluate their moral behavior as active decision-makers. Existing benchmarks, however, largely ask models to judge human actions from third-person textual vignettes, leaving untested how models behave when they themselves become the acting agent. We introduce ABHBench, a multimodal benchmark built from Detroit: Become Human, an interactive narrative game, to evaluate how foundation models make moral decisions in first-person human-android dilemmas. ABHBench places foundation models in branching narrative scenarios with visual observations, dialogue, memory, and action choices. Inspired by Asimov's Three Laws of Robotics, our framework measures Harmlessness, Obedience, and Self-preservation to capture the core tensions in human-AI relations, along with Autonomy, Stability, and Certainty to assess decision-making patterns. Across 12 foundation models, we find that moral behavior changes substantially under situated agentic context: 75% of models shift significantly between a text-only abstraction and the matched multimodal DBH scenario, and reasoning-enhanced models show no monotonic gains in consistency or human-oriented behavior. These results highlight the necessity of evaluating moral behavior in situated agentic contexts rather than relying on third-person judgment alone.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!