.png){kind=icon}
返回
A Fast Randomized Incremental Gradient Method for Decentralized Nonconvex Optimization
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2021.3122586.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this article, we study decentralized nonconvex finite-sum minimization problems described over a network of nodes, where each node possesses a local batch of data samples. In this context, we analyze a single-timescale randomized incremental gradient method, called GT-SAGA. GT-SAGA is computationally efficient as it evaluates one component gradient per node per iteration and achieves provably fast and robust performance by leveraging node-level variance reduction and network-level gradient tracking. For general smooth nonconvex problems, we show the almost sure and mean-squared convergence of GT-SAGA to a first-order stationary point and further describe regimes of practical significance, where it outperforms the existing approaches and achieves a network topology-independent iteration complexity, respectively. When the global function satisfies the Polyak-Lojaciewisz condition, we show that GT-SAGA exhibits linear convergence to an optimal solution in expectation and describe regimes of practical interest where the performance is network topology independent and improves upon the existing methods. Numerical experiments are included to highlight the main convergence aspects of GT-SAGA in nonconvex settings.
Keyword:
Convergence
Gradient methods
Optimization
Linear matrix inequalities
Stochastic processes
Robustness
Radio frequency
Decentralized nonconvex optimization
incremental gradient methods
variance reduction
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
.png){kind=icon}
分享收藏Effect of ω-3 fatty acids on rectal mucosal cell proliferation in subjects at risk for colon cancerΩ-3脂肪酸对结肠癌高危人群直肠黏膜细胞增殖的影响
OAAI
OAAI分享收藏A Geometrically Converging Dual Method for Distributed Optimization Over Time-Varying Graphs时变图上分布式优化的几何收敛对偶方法OAAI
OAAI分享收藏
查看更多内容