.png){kind=icon}
返回
Consensus-Based Cooperative Algorithms for Training Over Distributed Data Sets Using Stochastic Gradients
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2021.3071058.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this article, distributed algorithms are proposed for training a group of neural networks with private data sets. Stochastic gradients are utilized in order to eliminate the requirement for true gradients. To obtain a universal model of the distributed neural networks trained using local data sets only, consensus tools are introduced to derive the model toward the optimum. Most of the existing works employ diminishing learning rates, which are often slow and impracticable for online learning, while constant learning rates are studied in some recent works, but the principle for choosing the rates is not well established. In this article, constant learning rates are adopted to empower the proposed algorithms with tracking ability. Under mild conditions, the convergence of the proposed algorithms is established by exploring the error dynamics of the connected agents, which provides an upper bound for selecting the constant learning rates. Performances of the proposed algorithms are analyzed with and without gradient noises, in the sense of mean square error (MSE). It is proved that the MSE converges with bounded errors determined by the gradient noises, and the MSE converges to zero if the gradient noises are absent. Simulation results are provided to validate the effectiveness of the proposed algorithms.
Keyword:
Training
Neural networks
Convergence
Distributed databases
Machine learning algorithms
Optimization
Distributed algorithms
Consensus
convergence analysis
distributed training
multiagent systems
neural networks
optimization
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
.png){kind=icon}
Distributed Optimization for Multiagent Systems: An Edge-Based Fixed-Time Consensus Approach多代理系统的分布式优化: 基于边缘的固定时间共识方法
PREAI
分享收藏分享收藏Distributed Continuous-Time Algorithms for Resource Allocation Problems Over Weight-Balanced Digraphs权重平衡有向图上资源分配问题的分布式连续时间算法
PREAI
分享收藏
查看更多内容