.png){kind=icon}
返回
Online Distributed Stochastic Gradient Algorithm for Nonconvex Optimization With Compressed Communication
.png){kind=icon}
PRE
AI
DOI：10.1109/TAC.2023.3327183.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This article examines an online distributed optimization problem over an unbalanced digraph, in which a group of nodes in the network tries to collectively search for a minimizer of a time-varying global cost function while data are distributed among computing nodes. As the problem size becomes large, it will inevitably suffer from the communication bottleneck since each node that exchanges messages potentially transmits large amounts of information to its neighbors. To handle the issue, we design an online stochastic gradient algorithm with compressed communication when the knowledge of the gradient is available. We obtain the regret bounds for both nonconvex and convex cost functions, which can reach almost the same order of classic distributed optimization algorithms with exact communication. To resolve the scenario when the information of gradients is not accessible, a bandit version of the previous algorithm is then proposed. Explicit regret bounds of the bandit algorithm are also established for both nonconvex and convex cost functions. The result reveals that the performance of the bandit-feedback method is almost close to that of the gradient-feedback method. Several numerical experiments corroborate the main theoretical findings obtained in this article and exemplify a remarkable speedup when compared to existing distributed algorithms with exact communication.
Keyword:
Bandit-feedback
compressed communication
distributed optimization (DO)
nonconvex optimization
online optimization
stochastic approximation
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
The dose can make the poison: lessons learned from adverse in vivo toxicities caused by RNAi overexpression.png)
Silence
OAAI
SilenceIF0
OAAI分享收藏分享收藏
Distributed Projection Subgradient Algorithm Over Time-Varying General Unbalanced Directed Graphs时变一般不平衡有向图上的分布式投影次梯度算法
PREAI
分享收藏分享收藏
查看更多内容