Back to AI News
arxiv

Contrastive Discovery: Evaluating Competing Explanations in LLM-Driven Science

Contrastive Discovery is a method for LLM discovery agents that replaces pointwise hypothesis testing with contrast sets: scientific questions paired with mutually exclusive and collectively exhaustive explanations. Experiments are scored by how their evidence redistributes belief across the set, and a question is resolved when belief concentrates on one explanation. Across six domains, contrast sets resolved 73% of high-belief-shift pointwise rejections to a specific alternative, and search guided by contrast sets resolved 67% more questions than pointwise surprisal under the same experiment budget. In a blinded study, 38 human experts matched the agent's explanation on 78% of resolved questions.

AI Brief

It offers a concrete method for making LLM discovery agents resolve competing scientific explanations rather than only rejecting hypotheses, with quantitative evidence and expert validation.

arXiv:2611.14514v1 Announce Type: new Abstract: LLM agents are increasingly used for open-ended, data-driven scientific discovery, yet most of them evaluate hypotheses pointwise, testing an empirical claim $h$ only against its complement $\neg h$. When evidence rejects $h$, the agent learns that one explanation is unlikely but not which alternative the evidence supports. To recover that alternative, we introduce Contrastive Discovery, which replaces the single hypothesis with a contrast set, a scientific question paired with mutually exclusive and collectively exhaustive competing explanations. Each experiment is scored by how its evidence redistributes belief across the set, and the question is resolved when belief concentrates on one explanation. Because contrast sets change only the unit an agent proposes and scores, an existing agent can use them to re-evaluate evidence it has already collected or to guide its search. Across six domains, $89\%$ of the pointwise candidates with the largest belief shifts are rejections. On the same evidence, contrast sets resolve $73\%$ of these candidates to a specific alternative. When contrast sets guide search, the agent resolves $67\%$ more questions than pointwise surprisal under the same experiment budget. In a blinded study, $38$ human experts chose the same explanation on $78\%$ of resolved questions and on $82\%$ of questions where the pointwise agent rejected its hypothesis and stopped. Together, these results support explicit competing explanations as a way for discovery agents to resolve more scientific questions from the same evidence and validation budget.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!