DINOv2 Adaptation Generalizes Better Than Task-Specific Medical Segmentation Models
A new arXiv study compares adapting the DINOv2 foundation model with training U-Nets and ResU-Nets from scratch for medical image segmentation across colonoscopy, breast ultrasound and dermoscopy. The adapted foundation model generalized better on every independent external dataset, while task-specific models remained competitive in-distribution when training data were abundant and delineated boundaries more precisely. Only a small fraction of the foundation model needed adaptation, with the required capacity determined by the imaging domain rather than the tuning method; the authors also found that a widely used polyp test set duplicates images from another benchmark.