.png){kind=icon}
返回
Dominating Set Model Aggregation for communication-efficient decentralized deep learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2023.11.057.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Decentralized deep learning algorithms leverage peer-to-peer communication of model parameters and/or gradients over communication graphs among the learning agents with access to their private data sets. The majority of the studies in this area focus on achieving high accuracy, with many at the expense of increased communication overhead among the agents. However, large peer-to-peer communication overhead often becomes a practical challenge, especially in harsh environments such as for an underwater sensor network. In this paper, we aim to reduce communication overhead while achieving similar performance as the state-of-the-art algorithms. To achieve this, we use the concept of Minimum Connected Dominating Set from graph theory that is applied in ad hoc wireless networks to address communication overhead issues. Specifically, we propose a new decentralized deep learning algorithm called minimum connected Dominating Set Model Aggregation (DSMA). We investigate the efficacy of our method for different communication graph topologies with a small to large number of agents using varied neural network model architectures. Empirical results on benchmark data sets show a significant (up to 100X) reduction in communication time while preserving the accuracy or in some cases, increasing it compared to the state-of-the-art methods. We also present an analysis to show the convergence of our proposed algorithm.
Keyword:
Distributed deep learning
Communication-efficient
Convergence rate
Connected dominating set
Decentralized learning
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏 没有更多内容
没有更多内容