.png){kind=icon}
.png){kind=icon}
Return
FedNCV: Optimizing Federated Learning With Networked Control Variates
.png){kind=icon}
PRE
AI
DOI：10.1002/ett.70287.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Federated learning (FL), as an advanced distributed learning paradigm, faces significant challenges, particularly in terms of slow convergence and instability, which are exacerbated by heterogeneous data distributions. A critical issue in this context is data heterogeneity, which increases gradient estimation variance and drives the model toward local minima that are distant from the global optimum. Previous studies have primarily focused on using Control Variates (CVs) to reduce gradient estimate variance without introducing bias. In this work, we propose a novel distributed gradient optimization framework, FedNCV, aimed at effectively reducing gradient variance. Central to this approach is the use of the REINFORCE Leave-One-Out (RLOO), a CV-based technology, which serves as the core gradient estimator for FedNCV at both the client and server levels. We have developed an algorithm based on FedNCV and provided three theoretical results. Experimental evaluations demonstrate that the proposed method enhances performance. The dual structure of FedNCV equips it to address the challenges of data heterogeneity and scalability in federated networks, offering a promising solution for applications in heterogeneous FL environments. Additionally, the efficacy of FedNCV was validated across four diverse datasets under a Dirichlet distribution with , setting new performance benchmarks when compared to six leading methods.
Keywords:
control variates
federated learning
gradient optimization
Journal
IF:
2.5
Papers:
450
Citations:
3.9K