ETAPS: Gradual Pruning with Sparse Adaptation Achieves High Quality at Extreme Sparsity
Researchers propose ETAPS, a pruning method that gradually tightens a global sparsity constraint while training a sparse delta with a small fixed budget. This approach avoids model-sized optimizer states, enabling effici...