.png){kind=icon}
返回
A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization
.png){kind=icon}
OA
AI
DOI：10.1109/JAS.2022.105554.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered. This problem is an important component of many machine learning techniques with data parallelism, such as deep learning and federated learning. We propose a distributed primal-dual stochastic gradient descent (SGD) algorithm, suitable for arbitrarily connected communication networks and any smooth (possibly nonconvex) cost functions. We show that the proposed algorithm achieves the linear speedup convergence rate O(1/root nT) for general nonconvex cost functions and the linear speedup convergence rate O(1/nT)) when the global cost function satisfies the Polyak-Lojasiewicz (P-L) condition, where T is the total number of iterations. We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum. We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
Keyword:
Distributed nonconvex optimization
linear speedup
Polyak-Lojasiewicz (P-L) condition
primal-dual algorithm
stochastic gradient descent
期刊
I
IF:
19.2
论文数:
1.4K
被引数:
1.1W
机构
引用论文
.png){kind=icon}
分享收藏Phosphine-catalyzed dimerization of activated alkenes under ambient and high pressure conditions
PREAI
分享收藏Challenges and opportunities at the crossroads of Environmental Sustainability and Economy research
PREAI
分享收藏
查看更多内容