Research
DPA: Decentralized Primal Averaging with Quasi-Global Momentum for Heterogeneous Data
This paper introduces Decentralized Primal Averaging (DPA), a decentralized learning algorithm that combines primal averaging with a local quasi-global momentum buffer to handle highly heterogeneous data. It establishes convergence bounds for smooth nonconvex objectives and identifies conditions for linear speedup. Experiments on the GUT benchmark show DPA outperforms several baselines, especially under high heterogeneity on sparse ring topologies, with code publicly available.
decentralized trainingfederated learningoptimization algorithms