.png){kind=icon}
Return
A Distributed Stochastic Proximal-Gradient Algorithm for Composite Optimization
.png){kind=icon}
PRE
AI
DOI：10.1109/TCNS.2021.3065653.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In this article, we consider distributed composite optimization problems involving a common non-smooth regularization term over an undirected and connected network. Inspired by vast applications of this kind of problem in large-scale machine learning, the local cost function of each node is further set as an average of a certain amount of local cost subfunctions. For this scenario, most existing solutions based on the proximal method tend to ignore the cost of gradient evaluations, which results in degraded performance. We instead develop a distributed stochastic proximal-gradient algorithm to tackle the problems by employing the local unbiased stochastic averaging gradient method. At each iteration, only a single local cost subfunction is demanded by each node to evaluate the gradient, then the average of the latest stochastic gradients serves as the approximation of the true local gradient. An explicit linear convergence rate of the proposed algorithm is established with constant dual step-sizes for strongly convex local cost subfunctions with Lipschitz-continuous gradients. Furthermore, it is shown that, in the smooth cases, our simplified analysis technique can be extended to some notable primal-dual domain algorithms, such as DSA, EXTRA, and DIGing. Numerical experiments are presented to confirm the theoretical findings.
Keywords:
Linear convergence
linear convergence distributed composite optimization
machine learning
proximal-gradient method
stochastic averaging gradient (SAG)
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
5
Papers:
1.7K
Citations:
5.8K
Organization
View more organizations
Cited Papers
.png){kind=icon}
Effect of ω-3 fatty acids on rectal mucosal cell proliferation in subjects at risk for colon cancer
OAAI
OAAIShareSaveShareSaveShareSave
View more