arXiv:2611.06259v1 Announce Type: new
Abstract: People often make decisions by holistically reasoning over heterogeneous forms of personal information. In this paper, we study the relatively underexplored capability of multimodal personalized reasoning in multimodal large language models (MLLMs). To this end, we introduce \texttt{MPRBench}, a novel and comprehensive benchmark specifically designed to evaluate personalized reasoning across a wide range of tasks and real-world challenges. \texttt{MPRBench} consists of 12 sub-tasks and additionally supports interpretable analysis of representative error cases. Through extensive experiments, we find that current personalized MLLMs still struggle with personalized reasoning due to several key challenges, including inaccurate recognition of personalized concepts and sensitivity to irrelevant personal information. We hope that MPRBench will stimulate further research on personalized reasoning in MLLMs. Code and data are available at https://github.com/kkdh15/MPRBench
arxiv
MPRBench: A New Benchmark for Multimodal Personalized Reasoning in Vision-Language Models
MPRBench is a comprehensive benchmark introduced to evaluate multimodal personalized reasoning in large language models, covering 12 sub-tasks and supporting interpretable error analysis. Experiments show current MLLMs struggle with recognizing personalized concepts and are sensitive to irrelevant personal information. The benchmark aims to drive further research in this area.
Provides a concrete, recent benchmark with findings that highlight current limitations in MLLM personalized reasoning, valuable for researchers and developers.
Comments & Questions (0)
No comments yet
Be the first to comment!