.png){kind=icon}
返回
Distributed Stochastic Gradient Tracking Algorithm With Variance Reduction for Non-Convex Optimization
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2022.3170944.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This article proposes a distributed stochastic algorithm with variance reduction for general smooth non-convex finite-sum optimization, which has wide applications in signal processing and machine learning communities. In distributed setting, a large number of samples are allocated to multiple agents in the network. Each agent computes local stochastic gradient and communicates with its neighbors to seek for the global optimum. In this article, we develop a modified variance reduction technique to deal with the variance introduced by stochastic gradients. Combining gradient tracking and variance reduction techniques, this article proposes a distributed stochastic algorithm, gradient tracking algorithm with variance reduction (GT-VR), to solve large-scale non-convex finite-sum optimization over multiagent networks. A complete and rigorous proof shows that the GT-VR algorithm converges to the first-order stationary points with O(1/k) convergence rate. In addition, we provide the complexity analysis of the proposed algorithm. Compared with some existing first-order methods, the proposed algorithm has a lower O(PM epsilon b;(1)) gradient complexity under some mild condition. By comparing state-of-the-art algorithms and GT-VR in numerical simulations, we verify the efficiency of the proposed algorithm.
Keyword:
Signal processing algorithms
Optimization
Convergence
Random variables
Distributed algorithms
Machine learning
Distributed databases
Complexity analysis
distributed algorithm
non-convex finite-sum optimization
stochastic gradient
variance reduction
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
.png){kind=icon}
分享收藏Effect of ω-3 fatty acids on rectal mucosal cell proliferation in subjects at risk for colon cancerΩ-3脂肪酸对结肠癌高危人群直肠黏膜细胞增殖的影响
OAAI
OAAI分享收藏
查看更多内容