Stanford and UC Berkeley Release TokenGate for Learned Token Pruning in Long-Context LLM Prefill
Researchers at Stanford and UC Berkeley have released TokenGate, a drop-in module that adds learned token pruning to pretrained LLMs so long prompts are processed faster. A small MLP gate scores tokens at a chosen layer,...