.png){kind=icon}
返回
Self-adaptive asynchronous federated optimizer with adversarial sharpness-aware minimization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2024.07.045.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The past years have witnessed the success of a distributed learning system called Federated Learning (FL). Recently, asynchronous FL (AFL) has demonstrated its potential in concurrency compared to mainstream synchronous FL. However, the inherent systematic and statistical heterogeneity has presented several impediments to AFL: On the client side, the discrepancies in trips and local model drift impede global performance enhancement; On the server side, dynamic communication leads to significant fluctuations in gradient arrival time, while asynchronous arrival gradients with ambiguous value are not fully leveraged. In this paper, we propose an adaptive AFL framework, ARDAGH, which systematically addresses the aforementioned challenges: Firstly, to address the discrepancies in client trips, ARDAGH ensures their convergence by incorporating only 1-bit feedback information into the downlink. Secondly, to counter the drift of clients, ARDAGH generalizes the local models by employing our novel adversarial sharpness-aware minimization, which does not necessitate reliance on additional global variables. Thirdly, in the face of gradient latency issues, ARDAGH employs a communication-aware dropout strategy to adaptively compress gradients to ensure similar transmission times. Finally, to fully unleash the potential of each gradient, we establish a consistent optimal direction by conceptualizing the aggregation as an optimizer with successive momentum. In light of the comprehensive solution provided by ARDAGH, an algorithm named FedAMO is derived, and its superiority is confirmed by experimental results obtained under challenging prototype and simulation settings. Particularly in typical sentiment analysis tasks, FedAMO demonstrates an improvement of up to 5.351% with a 20.056-fold acceleration compared to conventional asynchronous methods.
Keyword:
Federated learning
Asynchronous aggregation
Self-adaptive system
Adversarial sharpness-aware minimization
期刊
F
IF:
6.1
论文数:
6.9K
被引数:
2.3W
机构
引用论文
Analysis and assessment of ship collision accidents using Fault Tree and Multiple Correspondence Analysis基于故障树和多重对应分析的船舶碰撞事故分析与评估
.png){kind=icon}
PREAI
分享收藏Towards Efficient and Stable K-Asynchronous Federated Learning With Unbounded Stale Gradients on Non-IID Data在非IID数据上实现具有无界陈旧梯度的高效稳定的K异步联合学习
OAAI
OAAI分享收藏A Triple-Step Asynchronous Federated Learning Mechanism for Client Activation, Interaction Optimization, and Aggregation Enhancement用于客户端激活，交互优化和聚合增强的三步异步联合学习机制
PREAI
分享收藏FedSA: A Semi-Asynchronous Federated Learning Mechanism in Heterogeneous Edge ComputingFedSA: 异构边缘计算中的半异步联邦学习机制OAAI
OAAI分享收藏分享收藏Effects of intravenous calcium on release of serotonin into jejunal lumen and portal circulation
PREAI
分享收藏
查看更多内容