A new algorithmic approach for finding flat minima: critically damped Polyak stepsizes
The paper introduces critically damped Polyak (CDP) stepsizes, an adaptive gradient-descent rule that multiplies the classical Polyak stepsize by 4 and a damping factor to bias training toward flatter minima. It provides...