Tsinghua Open-Sources AKV, a KV Cache Engineering Server for LLM Agents on SGLang
Researchers from Tsinghua University and CUHK have open-sourced AKV, an Apache-2.0 serving system that lets LLM agents drop and reposition KV cache entries instead of summarizing text history. AKV extends SGLang's r...