.png){kind=icon}
返回
Robust communication-efficient decentralized learning with heterogeneity
.png){kind=icon}
PRE
AI
DOI：10.1016/j.sysarc.2023.102900.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this paper, we propose a robust communication-efficient decentralized learning algorithm, named RCEDL, to address data heterogeneity, communication heterogeneity and communication efficiency simultaneously in real -world scenarios. To the best of our knowledge, this is the first work to address the above challenges in a united framework. In detail, we design a compressed cross-gradient aggregation mechanism with delay to resolve the Non-IID issues, a blocking-resilient mechanism which allows receiving delayed parameters and gradients, and a communication-efficient mechanism including parameters compression and adaptive neighbors selection methods to reduce the communication cost as much as possible. In addition, we also provide convergence analysis of RCEDL and prove its convergence rate O( & RADIC;1NK ) same with the state-of-the-art decentralized learning algorithms. Finally, we conduct extensive experiments to evaluate RCEDL algorithm on two widely used datasets CIFAR-10 and MNIST under different experimental settings. Compared with the state-of-the-art baseline methods, the proposed RCEDL is much more robust with higher accuracy and at least 3.4 x communication cost reduction under the heterogeneous environment.
Keyword:
Decentralized learning
Communication-efficient
Heterogeneity
期刊
IF:
4.1
论文数:
3.0K
被引数:
4.2K
机构
引用论文
A survey of federated learning for edge computing: Research problems and solutions面向边缘计算的联邦学习综述: 研究问题与解决方案
OAAI
.png){kind=icon}
OAAI分享收藏A federated deep learning framework for privacy preservation and communication efficiency一种用于隐私保护和通信效率的联合深度学习框架OAAI
OAAI分享收藏Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT基于Transformer的学习图结构在物联网多变量时间序列异常检测中的应用
OAAI
OAAI分享收藏
查看更多内容