Back to AI News
arxiv

DINOv2 Adaptation Generalizes Better Than Task-Specific Medical Segmentation Models

A new arXiv study compares adapting the DINOv2 foundation model with training U-Nets and ResU-Nets from scratch for medical image segmentation across colonoscopy, breast ultrasound and dermoscopy. The adapted foundation model generalized better on every independent external dataset, while task-specific models remained competitive in-distribution when training data were abundant and delineated boundaries more precisely. Only a small fraction of the foundation model needed adaptation, with the required capacity determined by the imaging domain rather than the tuning method; the authors also found that a widely used polyp test set duplicates images from another benchmark.

AI Brief

It provides concrete evidence on when foundation-model adaptation is worth the cost for clinical imaging, especially for cross-site generalization.

arXiv:2611.14519v1 Announce Type: cross Abstract: Foundation models pretrained on large natural-image collections are increasingly adapted for medical image segmentation, in the expectation that they will replace models trained for each task. Most comparisons, however, use data from the training distribution and a single task-specific baseline whose configuration is rarely justified, and therefore provide limited evidence of performance at new clinical sites. It also remains unclear how much of a foundation model must be adapted, and what determines this. We adapted a foundation model, DINOv2, with strategies ranging from tuning only its bias terms to full fine-tuning, and compared it with U-Nets and ResU-Nets trained from scratch. Experiments covered colonoscopy, breast ultrasound and dermoscopy, with testing on held-out data from the training source and on independent datasets from other institutions. The adapted foundation model generalised better on every external dataset. Within the training distribution, the task-specific models remained competitive when training data were abundant, and delineated boundaries more precisely, while the foundation model more often missed small lesions. Only a small fraction of the foundation model required adaptation, and the capacity needed was determined by the imaging domain rather than by the adaptation method. A widely used polyp test set was also found to duplicate images from another benchmark. Foundation-model adaptation is therefore warranted where models must work across sites, with capacity selected on the target domain.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!