Research
Hx2Dx ARENA benchmarks LLMs on interactive clinical diagnosis
Researchers introduce Hx2Dx ARENA, an interactive benchmark of 1,000 clinical cases from the New England Journal of Medicine that simulates encounters from initial presentation through history taking, physical examination and diagnostic workup. Open- and closed-source LLMs are evaluated under multiple configurations. Every evaluated model loses diagnostic accuracy in the interactive setting compared with a static setting where the full case is presented upfront, and the ability to ask targeted questions accounts for much of the variation between models and the accuracy loss. The work highlights a gap between reasoning over available clinical data and hypothesis-driven information gathering.
LLMsclinical diagnosisbenchmarking