.png){kind=icon}
Return
Distributed Stochastic Optimization Under Heavy-Tailed Noises
.png){kind=icon}
PRE
AI
DOI：10.1109/TAC.2025.3617572.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This article studies the distributed optimization problem in the presence of heavy-tailed gradient noises. Here, a heavy-tailed noise $\xi$ does not necessarily adhere to the bounded variance assumption, i.e., ${\mathbb {E}}[\Vert \xi \Vert ^{2}]\leq \nu ^{2}$ for some positive constant $\nu$. Instead, it satisfies a more general condition ${\mathbb {E}}[\Vert \xi \Vert ^\delta ]\leq \nu ^\delta$ for some $1< \delta \leq 2$. The commonly used bounded variance assumption is a special case of the considered noise assumption. While several distributed optimization algorithms have been proposed for scenarios involving heavy-tailed noise, these algorithms need a centralized server in the network, which collects the information of all clients. Different from these algorithms, this article considers a scenario where there is no centralized server and the agents can only exchange information with neighbors within a communication graph. A distributed method combining gradient clipping and distributed stochastic gradient projection is proposed. It is proven that when the gradient descent step-size and the gradient clipping step-size meet certain conditions, the state of each agent converges to an optimal solution of the distributed optimization problem with probability 1. The simulation results validate the algorithm.
Keywords:
Distributed stochastic optimization
heavy-tailed noise
infinite variance data
multiagent system
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSaveShareSave
View more