Latest AI updates

MedPhysBench: Benchmarking LLMs on Medical Physics Calculations and Error Detection

MedPhysBench is a new benchmark with 1,200 held-out items testing language models on medical physics calculations and error detection across radiation therapy, imaging, nuclear medicine, and radiation protection. Twelve models scored 33.3–100% on calculations, with top models near-perfect but still making errors, and several models failing to catch planted errors like an omitted inverse-square correction. The benchmark includes code, per-item outcomes, and targeted test responses.

Why it matters: Provides a concrete, recent evaluation of LLM performance in a high-stakes medical domain, revealing both strengths and critical failure modes.