.png){kind=icon}
返回
Communication Compression for Decentralized Learning With Operator Splitting Methods
.png){kind=icon}
OA
AI
DOI：10.1109/TSIPN.2023.3307894.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In decentralized learning, operator splitting methods using a primal-dual formulation (e.g., Edge-Consensus Learning (ECL)) have been shown to be robust to heterogeneous data and have attracted significant attention in recent years. However, in the ECL, a node needs to exchange dual variables with its neighbors. These exchanges incur significant communication costs. For the Gossip-based algorithms, many compression methods have been proposed, but these Gossip-based algorithms do not perform well when the data distribution held by each node is statistically heterogeneous. In this work, we propose a novel framework of the compression methods for the ECL, called the Communication Compressed ECL (C-ECL). Specifically, we reformulate the update formulas of the ECL and propose to compress the update values of the dual variables. We demonstrate experimentally that the C-ECL can achieve a nearly equivalent performance with fewer parameter exchanges than the ECL. Moreover, we demonstrate that the C-ECL is more robust to heterogeneous data than the Gossip-based algorithms.
Keyword:
Douglas-Rachford splitting
operator splitting
decentralized learning
期刊
IF:
4.9
论文数:
733
被引数:
1.9K
机构
引用论文
Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMMDouglas-Rachford分裂和ADMM的线性收敛和度量选择
OAAI
OAAI分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
查看更多内容