.png){kind=icon}
返回
Communication-Efficient Federated Learning with Adaptive Quantization
.png){kind=icon}
OA
AI
DOI：10.1145/3510587.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Federated learning (FL) has attracted tremendous attentions in recent years due to its privacy-preserving measures and great potential in some distributed but privacy-sensitive applications, such as finance and health. However, high communication overloads for transmitting high-dimensional networks and extra security masks remain a bottleneck of FL. This article proposes a communication-efficient FL framework with an Adaptive Quantized Gradient (AQG), which adaptively adjusts the quantization level based on a local gradient's update to fully utilize the heterogeneity of local data distribution for reducing unnecessary transmissions. In addition, client dropout issues are taken into account and an Augmented AQG is developed, which could limit the dropout noise with an appropriate amplification mechanism for transmitted gradients. Theoretical analysis and experiment results show that the proposed AQG leads to 18% to 50% of additional transmission reduction as compared with existing popular methods, including Quantized Gradient Descent (QGD) and Lazily Aggregated Quantized (LAQ) gradient-based methods without deteriorating convergence properties. Experiments with heterogenous data distributions corroborate a more significant transmission reduction compared with independent identical data distributions. The proposed AQG is robust to a client dropping rate up to 90% empirically, and the Augmented AQG manages to further improve the FL system's communication efficiency with the presence of moderate-scale client dropouts commonly seen in practical FL scenarios.
Keyword:
Federated learning
information compression
communication efficiency
期刊
IF:
6.6
论文数:
1.5K
被引数:
6.2K
机构
引用论文
.png){kind=icon}
分享收藏Predictive factors of restenosis after multivessel percutaneous transluminal coronary angioplasty多支血管经皮冠状动脉腔内成形术后再狭窄的预测因素
PREAI
分享收藏分享收藏Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues车辆物联网的联合学习: 最新进展和开放问题
OAAI
OAAI分享收藏
查看更多内容