Back to AI News
arxiv

Online Temporal Rotations for 2 Bit KV-Cache Quantization in Video Diffusion Transformers

The paper introduces Temporal Rotation Quantization (TRQ), which rotates temporally adjacent KV-cache vectors before quantization in video diffusion transformers. For each block of temporally adjacent cache vectors, TRQ fits an orthogonal transform that maps per-channel means toward a near-constant vector, parameterized as a tree of 2x2 planar rotations stored with two bits each plus a fixed shared Hadamard transform. The rotations are fit online per video without gradients or model weight changes, and the authors prove a greedy fit is globally optimal for their mean-flattening objective. At 2-bit precision and matched compression, TRQ improves VBench subject consistency over QVG from 90.38 to 92.02 on LongCat-Video-13B and from 86.33 to 87.61 on HY-WorldPlay-8B, with fused Triton kernels released for inference.

AI Brief

It offers a concrete, gradient-free 2-bit KV-cache quantization method with measured VBench gains and released kernels for long video generation.

arXiv:2611.26334v1 Announce Type: cross Abstract: Video diffusion transformers reuse key--value (KV) caches across denoising steps, which quickly becomes a bottleneck for long video generation. We find that quantizing temporally distant cache vectors leads to a large range and high quantization error, but that nearby video tokens in the KV-cache share a persistent pattern per channel. These patterns also vary across videos, motivating rotations that are adapted to each video's unique cache. We propose Temporal Rotation Quantization (TRQ), a method that rotates frames in the cache that are adjacent in time before quantization. For each block of temporally adjacent cache vectors, TRQ fits an orthogonal transform that maps the block's per-channel means to a vector that is close to a constant across channels. To keep the quantization format lightweight, we parameterize this rotation as a tree of $2 \times 2$ planar rotations, each stored with only two bits, followed by a fixed Hadamard transform shared across all blocks. The rotations are fit without gradients or changes to model weights, and we prove that a simple greedy fit is globally optimal for our 'mean-flattening' objective. The transforms are fit online for each new video, and we release fused Triton kernels to make this process efficient at inference. At 2-bit precision and matched compression, TRQ improves VBench subject consistency over the state-of-the-art method QVG from 90.38 to 92.02 on LongCat-Video-13B and from 86.33 to 87.61 on HY-WorldPlay-8B, as well as improving other metrics.
Be the first to rate this article.

Comments & Questions (0)

Captcha
Please be respectful — let's keep the conversation friendly.

No comments yet

Be the first to comment!