Back to AI News
arxiv

EcoVGGT Uses Pinned Keys for Efficient Long-Video 3D Reconstruction

EcoVGGT is a training-free inference framework for feed-forward multi-view transformers that uses a Pinned Key Set of 31 evenly spread frames to make cross-frame attention cost constant per frame. It includes an online stream for updating pinned frames during recording and alignment modules that flag and reduce failures by fine-tuning six middle global blocks. On supplied evaluations, EcoVGGT with pi^3 matches pi^3’s success rate on 1,000-frame ScanNet videos at 13.8 times its speed, while refocused EcoVGGT with VGGT raises VGGT’s success rate from 70% to 82% at 8.5 times its speed on 500 frames. The method also reconstructs 4,000 frames on one GPU and streams 1,000 frames at 18.6 frames per second with the lowest online absolute trajectory error.

AI Brief

It gives readers a concrete way to run long-video 3D reconstruction more efficiently and streamably without full retraining.

arXiv:2611.18099v1 Announce Type: cross Abstract: Feed-forward multi-view transformers such as VGGT recover the cameras and geometry of a video in one pass, but the cost of their cross-frame attention grows quadratically with the frame count, and their accuracy drops on long videos. Most remedies approximate attention over all frames within a budget or select a few keys by their content. We hypothesize that the keys of a long video can be chosen without looking at its content, and our experiments on VGGT and $\pi^3$ support it: a few dozen evenly spread frames can serve as the keys of every frame. We present EcoVGGT, a training-free inference framework whose keys form a Pinned Key Set of 31 such frames. A pin pass stores their tokens in a Scene Cache, and the other frames read it in independent batches that reproduce joint inference with these keys exactly, at a constant cost per frame. An Online Stream updates the pinned frames during recording. Six middle global blocks align the frames and form a reusable VGGT module: the Alignment Score reads their attention to flag failures, and Alignment Refocusing reduces them by fine-tuning only these blocks (3.6 GPU hours). EcoVGGT ($\pi^3$) matches the success rate of $\pi^3$ on 1,000-frame ScanNet videos at 13.8 times its speed. With refocusing, EcoVGGT (VGGT) raises the success rate of VGGT from 70% to 82% at 8.5 times its speed on 500 frames. EcoVGGT reconstructs 4,000 frames on one GPU, and EcoVGGT ($\pi^3$) streams 1,000 frames at 18.6 frames per second with the lowest online absolute trajectory error.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!