KV Cache Reuse Across Model Updates Preserves Task Utility but Diverges From Native Behavior
This arXiv paper studies whether key-value caches computed by one model version can be reused after weights are updated, examining four model updates with a detailed case study of DeepSeek-V4-Flash to Flash-0731. The authors find caches can retain useful task information without faithfully reproducing the updated model's behavior, with retrieval accuracy ranging from near-native performance to large losses. Where source caches already exist, reuse yields median time-to-first-token speedups of 2.10x to 5.08x on three longer-prefix QA benchmarks. They introduce Functional Cache Divergence, comparing next-token distributions under reused and native caches, and show compatibility is better for natural-text continuation than for single-token retrieval as prefix length grows.