.png){kind=icon}
返回
Distributed Stochastic Optimization Under a General Variance Condition
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2024.3393169.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Distributed stochastic optimization has drawn great attention recently due to its effectiveness in solving large-scale machine learning problems. Although numerous algorithms have been proposed and successfully applied to general practical problems, their theoretical guarantees mainly rely on certain boundedness conditions on the stochastic gradients, varying from uniform boundedness to the relaxed growth condition. In addition, how to characterize the data heterogeneity among the agents and its impacts on the algorithmic performance remains challenging. In light of such motivations, we revisit the classical federated averaging algorithm (McMahan et al., 2017) as well as the more recent SCAFFOLD method (Karimireddy et al., 2020) for solving the distributed stochastic optimization problem and establish the convergence results under only a mild variance condition on the stochastic gradients for smooth nonconvex objective functions. Almost sure convergence to a stationary point is also established under the condition. Moreover, we discuss a more informative measurement for data heterogeneity as well as its implications.
Keyword:
Optimization
Linear programming
Distributed databases
Gradient methods
Convergence
Complexity theory
Particle measurements
Distributed optimization
nonconvex optimization
stochastic optimization
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏分享收藏分享收藏
分享收藏
查看更多内容