Back to AI News
arxiv

Know When to Zoom: How Small Is Too Small for Vision-Language Models?

A new arXiv paper measures how much visual detail survives when online vision-language services shrink or reject large images, testing seven models from four vendors with a printed-code "eye exam" and 601 questions about small objects in real photos. The smallest text a model reads half the time grows from 6 pixels or less on small images to a predicted 10-13 pixels on a 7680x4320 photo at default settings. Splitting photos into up to nine tiles raises accuracy by 11.1 to 14.5 percentage points for Claude and Gemini and, by avoiding rejections, lifts GPT-6.1 Sol from 50.7% to 83.7%; a two-step zoom where the model names the region it needs matches or beats tiling on Gemini at about 40% of the image tokens. The authors also report differing failure modes when detail is lost, and release the exam, all model answers and code alongside five rules for when an agent should zoom.

AI Brief

It gives concrete, vendor-specific measurements of how image downscaling degrades VLM accuracy and quantifies cheaper zoom/tiling strategies teams can apply to agent pipelines.

arXiv:2611.22709v1 Announce Type: cross Abstract: Online vision-language models often miss small details in large images, because the service shrinks the image, or rejects it, before the model ever sees it. A car 20 pixels wide in an 8000 by 6000 drone photo reaches Claude about 6 pixels wide, and OpenAI's default rejects the photo outright. We measure how much detail survives for seven models from four vendors, using an eye exam of printed codes and 601 questions about small objects in real photos. The smallest text a model reads half the time grows from 6 pixels or less on a small image to a predicted 10 to 13 pixels on a 7680 by 4320 photo at default settings. Zooming wins much of the detail back. Splitting each photo into up to nine tiles raises accuracy by 11.1 to 14.5 percentage points for Claude and Gemini, and by avoiding rejections it lifts GPT-6.1 Sol from 50.7% to 83.7%. A two-step zoom, in which the model first names the region it needs, matches or beats tiles on Gemini and comes within 2.5 points of them on Claude, with about 40% of their image tokens. When a detail is lost, Claude and Gemini both say they cannot tell about half the time, but they fail differently in the rest: Gemini gives a confident wrong answer in 43% to 48% of these cases, Claude in only 5% to 9%. A check based on detail size, run before the call, misses most of the gain from tiles on Claude and Gemini, but Claude's own reply is a reliable trigger: tiling only after an unsure reply comes within 1.4 points of always tiling with 62% to 70% of the image tokens. We turn these findings into five rules for when an agent should zoom, and release the eye exam, every model answer and code at https://siddvoh.com/know-when-to-zoom.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!