Back to AI News
arxiv

Motion-Aware Reward Model Targets 'Motion Collapse' in Video Editing

Researchers identify a failure mode called motion collapse in reward models for instruction-guided video editing: on VE-Bench, a composite reward preferred a motion-suppressed candidate in 97% of cases, and learned video rewards and a VLM judge also struggled with mild suppression. They propose a motion-aware reward model that adds an RGB frame-difference stream and preference training against motion-suppressed and fabricated-motion negatives, with a distillation anchor to preserve alignment with human judgments. The model cut selection of motion-suppressed candidates to 0% on the motion ladder while reaching a Spearman correlation of 0.57 with human quality ratings, detected unseen suppression variants, and preserved motion during reward-guided fine-tuning on two editing pipelines.

AI Brief

It exposes a concrete blind spot in video-editing reward models and offers a tested architectural fix, which matters to teams using rewards for video generation or editing pipelines.

arXiv:2611.22495v1 Announce Type: cross Abstract: Reward models for instruction-guided video editing can favor temporal consistency at the expense of motion. We study this failure, termed \emph{motion collapse}, using controlled motion-suppression variants of real edits. On VE-Bench, the evaluated composite reward prefers a motion-suppressed candidate in $97\%$ of cases. Learned video rewards and a VLM judge also struggle with mild suppression, while latent optimization and reward-guided fine-tuning exploit this weakness to increase reward as motion declines. A linear probe of our base reward backbone performs near chance on mild suppression, motivating an explicit inter-frame dynamics input. We propose a motion-aware reward model that combines an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives. A distillation anchor preserves alignment with human quality judgments. On the VE-Bench motion ladder, our model reduces the rate of selecting motion-suppressed candidates to $0\%$ while achieving a Spearman correlation of $0.57$ with human quality ratings. It also detects unseen suppression variants and preserves motion during reward-guided fine-tuning on two editing pipelines. These results support explicitly modeling motion in rewards for video editing.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!