.png){kind=icon}
返回
Federated Adam-Type Algorithm for Distributed Optimization With Lazy Strategy
.png){kind=icon}
PRE
AI
DOI：10.1109/JIOT.2022.3175997.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
For large-scale machine learning tasks, distributing data in multiple clients, and using distributed optimization algorithms with a parameter server can accelerate the training process. The federated average algorithm has been widely used for distributed optimization via training local models in parallel and aggregating local models in a server to obtain the global model. To further improve the performance of the federated average algorithm, a novel federated learning algorithm have been proposed in this article by embedding a lazy strategy in the distributed Adam-type algorithm. In the proposed algorithm, the learning rate is adjusted adaptively in local update and lazy update strategy is applied on the second-order momentum of clients to make the learning rate identical. The convergence of the proposed algorithm is provided for both convex and nonconvex loss functions. Experiments have been conducted on MNIST digit recognition data set and CIFAR-10 data set. Experimental results show that the proposed algorithm can significantly reduce the communication overhead, thereby reduce the training time by 60% for CIFAR-10 data set, and the proposed algorithm achieve better performance than the federated average algorithm and its momentum version.
Keyword:
Training
Convergence
Adaptation models
Internet of Things
Logistics
Costs
Collaborative work
Adam
distributed optimization
federated learning
stochastic gradient descent (SGD)
期刊
IF:
8.9
论文数:
1.4W
被引数:
7.8W
机构
引用论文
Optimizing Federated Learning in Distributed Industrial IoT: A Multi-Agent Approach优化分布式工业物联网中的联邦学习: 一种多主体方法
OAAI
.png){kind=icon}
OAAI分享收藏分享收藏FedDetect: A Novel Privacy-Preserving Federated Learning Framework for Energy Theft Detection in Smart GridFedDetect: 一种新颖的隐私保护联邦学习框架，用于智能电网中的能量盗窃检测
PREAI
分享收藏AUCTION: Automated and Quality-Aware Client Selection Framework for Efficient Federated Learning
PREAI
分享收藏Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT物联网中无线边缘智能的高效通信联合学习
PREAI
分享收藏
分享收藏
查看更多内容