AURA streams audio-driven talking-head video with a rolling KV cache
AURA is an audio-driven video diffusion model that generates video chunk by chunk using a rolling KV cache, so it only needs the current chunk's audio instead of multi-second look-ahead. Two sizes are reported: AURA 1.3B streams 576p at 20.7 fps on a single GPU for a 16 Hz playback rate, while AURA 5B streams 720p at 9.7 fps. A two-stage distillation pipeline aligns a causal student with a bidirectional teacher, then applies Distribution Matching Distillation with backward simulation. The authors report AURA 1.3B runs 23x faster than its teacher and AURA 5B runs 17x faster while matching its TalkVerse teacher at 720p, and they release both models' weights plus 30K synthetic videos and precomputed teacher trajectories.
It offers a concrete low-latency streaming approach for interactive avatars and releases weights and data for reproducibility.
Comments & Questions (0)
No comments yet
Be the first to comment!