.png){kind=icon}
返回
FedProf: Selective Federated Learning Based on Distributional Representation Profiling
.png){kind=icon}
OA
AI
DOI：10.1109/TPDS.2023.3265588.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Federated Learning (FL) has shown great potential as a privacy-preserving solution to learning from decentralized data that are only accessible to end devices (i.e., clients). The data locality constraint offers strong privacy protection but also makes FL sensitive to the condition of local data. Apart from statistical heterogeneity, a large proportion of the clients, in many scenarios, are probably in possession of low-quality data that are biased, noisy or even irrelevant. As a result, they could significantly slow down the convergence of the global model we aim to build and also compromise its quality. In light of this, we first present a new view of local data by looking into the representation space and observing that they converge in distribution to Normal distributions before activation. We provide theoretical analysis to support our finding. Further, we propose FedProf, a novel algorithm for optimizing FL over non-IID data of mixed quality. The key of our approach is a distributional representation profiling and matching scheme that uses the global model to dynamically profile data representations and allows for low-cost, lightweight representation matching. Using the scheme we sample clients adaptively in FL to mitigate the impact of low-quality data on the training process. We evaluated our solution with extensive experiments on different tasks and data conditions under various FL settings. The results demonstrate that the selective behavior of our algorithm leads to a significant reduction in the number of communication rounds and the amount of time (up to 2.4x speedup) for the global model to converge and also provides accuracy gain.
Keyword:
Training
Distributed databases
Data models
Computational modeling
Servers
Task analysis
Convergence
Distributed systems
federated learning
machine learning
neural networks
representation learning
期刊
IF:
6
论文数:
5.2K
被引数:
1.1W
机构
引用论文
Study on energy-consumption regularities of cloud computing systems by a novel evaluation model基于新型评价模型的云计算系统能耗规律研究.png)
COMPUTING
COMPUTINGIF2.8
PREAI
分享收藏
查看更多内容