LoopVQ: Sub-1-bit KV Cache Quantization for Looped Transformers
LoopVQ is a vector quantization method for KV caches in looped transformers, exploiting redundancy along the loop axis and variable sensitivity to quantization. It achieves 0.5 bits per element, outperforming existing KV...