Query-Guided RoPE Modulation Targets Lost-in-the-Middle in Long-Context LLMs
Researchers propose QGPM, a training-free method that reshapes rotary position embeddings using query relevance derived from the model's own attention heads. It builds a broad relevance map for lower layers and a sh...