Sparse Text Diffusion under Dual Budgets: Context Connectivity and Expert Activation
The paper introduces DualBudget, a sparse text diffusion method that separately budgets attention graph edges and expert activations. It combines local, dilated, and global attention with conditional expert computation....