Back to AI News
arxiv

Agentic Retriever–Perceiver–Reasoner Framework for Multimodal Scientific Document Reasoning

The paper introduces an agentic Retriever–Perceiver–Reasoner framework for reasoning over scientific documents where evidence is spread across text, figures and tables. The Retriever selects source-aligned evidence under a context budget, the Perceiver builds a question-conditioned latent ontology over multimodal evidence, and the Reasoner combines that latent guidance with the original sources to produce grounded answers. The three components are trained via diagnostic-guided co-evolution, using a shared answering reward plus Retrieval Potential Feedback and Latent Efficacy Feedback to target resampling. Across four benchmarks and seven evaluation settings, the framework reports average scores of 69.9 and 75.0 with 7B and 8B Reasoner backbones, exceeding the strongest baseline by 4.0 and 9.1 points.

AI Brief

It offers a concrete method and benchmark results for improving multimodal retrieval and evidence use in scientific document reasoning agents.

arXiv:2611.22837v1 Announce Type: new Abstract: Scientific document reasoning requires an agent to acquire and integrate evidence distributed across text, figures, and tables. Retrieval can identify relevant sources while missing complementary support, and answer-level feedback alone does not reveal whether an error stems from evidence acquisition or evidence use. We present an agentic Retriever--Perceiver--Reasoner framework for multimodal scientific document reasoning. The Retriever sequentially selects source-aligned evidence under a context budget. Given the selected evidence and question, the Perceiver constructs a question-conditioned latent ontology, an implicit organization of multimodal evidence in a shared continuous space. The Reasoner combines this latent guidance with the original sources to produce grounded answers. We train the three components through diagnostic-guided co-evolution: alternating updates optimize them with a shared answering reward, while Retrieval Potential Feedback (RPF) and Latent Efficacy Feedback (LEF) identify questions that need better retrieval or latent guidance and direct targeted resampling. Across four benchmarks and seven evaluation settings, our framework obtains average scores of 69.9 and 75.0 with 7B and 8B Reasoner backbones, exceeding the strongest baseline by 4.0 and 9.1 points, respectively.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!