.png){kind=icon}
Return
SeqFedRPC: Sequential Federated Learning With Regularized Parameter Clustering
.png){kind=icon}
PRE
AI
DOI：10.1002/ett.70477.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Sequential federated learning (SFL) enables collaborative model training across clients in a chain manner, providing communication-efficient benefits over traditional all-gather parallel federated learning (PFL). However, SFL training often suffers from slow convergence and performance degradation due to nonidentically distributed (non-IID) data distribution. In motivation experiment, we find that model decoupling by partitioning the model into shared and personalized parameters and using just a few personalized parameters with large gradients can improve SFL training performance. Based on above findings, we propose SeqFedRPC, a novel model decoupling based SFL framework with regularized parameter clustering. We introduce a regularization term to promote parameter sparsification and amplify gradient differences, which aids in gradient-based parameter clustering. Then, we employ a clustering-based scheme to adaptively decouple the model parameters into shared and personalized subsets, thereby addressing the challenge of non-IID data by adapting global knowledge with shared parameters and client-specific distributions with personalized parameters. Extensive experiments on eight benchmark datasets demonstrate that SeqFedRPC surpasses eight SOTA methods, with each client personalizing less than 10% of the total parameters on all datasets at
.
Keywords:
parameter decoupling
regularized parameter clustering
sequential federated learning
Journal
IF:
2.5
Papers:
480
Citations:
3.9K
Organization
Cited Papers
.png){kind=icon}
ShareSave
ShareSaveShareSaveHeterogeneous Training Intensity for Federated Learning: A Deep Reinforcement Learning Approach
PREAI
ShareSave
View more