Back to AI News
arxiv

ZoomGain builds fine-grained visual supervision from global-to-zoom perception gaps

ZoomGain is a data construction pipeline that creates questions about a full image using visual evidence observable only after zooming in. It verifies answers on the zoomed view with multiple models, checks that questions remain unambiguous in the full image and cannot be answered from language priors, and generates multiple-choice distractors. The authors build ZoomGainBench, a 1,000-question bilingual benchmark where the best of 28 tested VLMs scores 61.08, and report that training on 9K samples improves 4B/9B models, with the 9B model reaching 90.25 on HR-Bench 4K.

AI Brief

It offers a concrete pipeline and benchmark for generating fine-grained visual supervision, with reported gains that may interest VLM researchers and practitioners.

arXiv:2611.14076v1 Announce Type: cross Abstract: Fine-grained visual perception requires Vision-Language Models (VLMs) to locate and interpret subtle evidence within complex scenes. Constructing supervision for this ability is challenging: questions must have correct, unambiguous answers that depend on actual fine-grained visual details rather than coarse cues or language priors. Verifying these properties makes manual annotation labor-intensive and automatic generation difficult for models that themselves struggle with fine-grained perception. Yet the same models can often recognize such details once the region is zoomed in. We exploit this global-to-zoom perception gap and introduce ZoomGain, a data construction pipeline that constructs questions about the full image from visual information gained only by zooming in. It verifies each answer on the zoomed view with multiple models, ensures that each question remains unambiguous in the full image and cannot be answered from language priors, and specifically generates high-quality distractors for multiple-choice questions. With this pipeline and human-only screening (no manual QA editing), we build ZoomGainBench, a 1,000-question bilingual benchmark on which the best of 28 VLMs achieves only 61.08. Failure analysis reveals that models favor a particular type of distractor when answering multiple-choice questions incorrectly. Beyond evaluation, training on just 9K ZoomGain samples substantially improves the visual perception capabilities of our 4B/9B models, enabling them to outperform several hundred-billion-parameter models across various perception benchmarks. The trained 9B model achieves SoTA performance on several benchmarks, such as 90.25 on HR-Bench 4K. These results establish global-to-zoom perception gaps as a practical basis for scaling fine-grained visual supervision for both evaluation and training. Our code is available at https://github.com/inclusionAI/ZoomGain.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!