Two-Clustering Regime of Token Dynamics in Causal Attention
The paper partially resolves and extends a conjecture on clustering in causal attention masking to time-dependent parameter regimes. It proves that two-cluster convergence holds when the largest eigenvalue of the value t...