Back to AI News
arxiv

VideoACE benchmark evaluates agentic video understanding with high-frame-rate actions

VideoACE is a new benchmark for agentic video understanding that tests whether models can use interaction to improve judgments. It contains 1,000 multiple-choice questions across six categories of fine-grained action understanding, with temporal evidence annotations and screened user hints. The evaluation environment provides video inspection tools and a user simulator, letting models adjust temporal coverage, sampling density, and spatial detail and revise judgments under feedback. Five models were evaluated across single-pass observation, autonomous tool use, and tool use with user feedback.

AI Brief

It offers a concrete testbed for measuring how video models exploit interactive tools and feedback rather than fixed observation.

arXiv:2611.24184v1 Announce Type: cross Abstract: Agentic video understanding requires models to actively interact with video environments to gather evidence and form judgments. Evaluating this capability therefore requires assessing whether models can translate opportunities for interaction into better understanding. We introduce VideoACE, a benchmark for assessing video understanding under conditions that support sustained interaction. We focus on high-frame-rate action videos, where rapid, subtle changes make targeted observation especially valuable. The benchmark comprises 1,000 multiple-choice questions spanning six categories of fine-grained action understanding, accompanied by temporal evidence annotations and user hints screened for answer leakage. Its evaluation environment combines video inspection tools with a user simulator, allowing models to adjust temporal coverage, sampling density, and spatial detail, and to revisit evidence and revise their judgments under simulated user feedback. We evaluate single-pass observation, autonomous tool use, and tool use combined with user feedback to examine how effectively models exploit these opportunities. Evaluation of five models reveals distinct patterns in how they acquire evidence and respond to opportunities for further interaction. These findings establish VideoACE as a testbed for examining agentic video understanding beyond performance under a fixed observation setting.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!