arXiv:2611.08369v1 Announce Type: new
Abstract: Current medical benchmarks typically break down clinical competence into isolated factual questions, single-image tasks, or static case vignettes. These tests score individual answers, but reveal less about whether a model can integrate subspecialty knowledge with multimodal evidence and track a patient's condition throughout a course of care. To close this gap, we introduce NPCBench, a clinical apprenticeship benchmark for nasopharyngeal carcinoma (NPC). NPCBench mirrors specialist training, from guideline and foundational knowledge to case planning, longitudinal review, CarePath integration, and open-ended consultation. It contains 4,777 component and consultation items and a 100-patient CarePath cohort with 417 scored diagnostic and care tasks. The episode-level evaluation tests whether diagnostic and care answers are jointly correct within a care pathway. By linking stepwise tasks to patient-level care trajectories, NPCBench evaluates whether models ground decisions in visual evidence, interpret changes in patient state, and meet clinical requirements in consultation. Across 30 evaluated models, we observe a composition gap: strong performance on individual tasks does not ensure jointly correct diagnostic and care answers for the same patient. Among 21 multimodal models, strict CarePath success reaches at most 23%. Paired controls and physician calibration further examine evidence grounding, longitudinal revision, and consultation judging. Overall, NPCBench broadens the evaluation of medical artificial intelligence beyond static question-answering paradigms to an apprenticeship-oriented assessment of clinical workflows, thereby offering a testbed for evidence-based and temporally coherent reasoning in oncology care.
arxiv
NPCBench: A New Benchmark for Multimodal Care-Pathway Reasoning in Nasopharyngeal Carcinoma
NPCBench is a clinical apprenticeship benchmark for nasopharyngeal carcinoma that evaluates AI models on integrated, longitudinal care-pathway reasoning rather than isolated questions. It includes 4,777 items and a 100-patient cohort with 417 scored tasks, testing evidence grounding and temporal coherence. Evaluations of 30 models reveal a composition gap: strong individual task performance does not guarantee jointly correct diagnostic and care answers. The benchmark aims to push medical AI beyond static QA toward workflow-oriented assessment.
This introduces a novel benchmark addressing a critical gap in medical AI evaluation, with concrete findings on model limitations in multimodal, longitudinal reasoning.
Comments & Questions (0)
No comments yet
Be the first to comment!