.png){kind=icon}
返回
Federated Learning With Lossy Distributed Source Coding: Analysis and Optimization
.png){kind=icon}
OA
AI
DOI：10.1109/TCOMM.2023.3277882.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, federated learning (FL), which replaces data sharing with model sharing, has emerged as an efficient and privacy-friendly machine learning (ML) paradigm. One of the main challenges in FL is the huge communication cost for model aggregation. Many compression/quantization schemes have been proposed to reduce the communication cost for model aggregation. However, the following question remains unanswered: What is the fundamental trade-off between the communication cost and the FL convergence performance? In this paper, we manage to answer this question. Specifically, we first put forth a general framework for model aggregation performance analysis based on the rate-distortion theory. Under the proposed analysis framework, we derive an inner bound of the rate-distortion region of model aggregation. We then conduct an FL convergence analysis to connect the aggregation distortion and the FL convergence performance. We formulate an aggregation distortion minimization problem to improve the FL convergence performance. Two algorithms are developed to solve the above problem. Numerical results on aggregation distortion, convergence performance, and communication cost demonstrate that the baseline model aggregation schemes still have great potential for further improvement.
Keyword:
Federated learning
model aggregation
rate-distortion theory
distributed source coding
Berger-Tung coding
majorization-minimization
期刊
IF:
8.3
论文数:
1.2W
被引数:
3.6W
机构
引用论文
.png){kind=icon}
分享收藏Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning用于空中联合边缘学习的时间结构辅助梯度聚合OAAI
OAAI分享收藏Temperature-Constrained Topology Optimization of Transient Heat Conduction Problems瞬态热传导问题的温度约束拓扑优化
PREAI
分享收藏
查看更多内容