FedNCV: Optimizing Federated Learning With Networked Control Variates

doi:10.1002/ett.70287

2025-11-04

PRE

DOI：10.1002/ett.70287

原文链接

原文求助

摘要

En 中文

联邦学习（FL）作为一种先进的分布式学习范式，面临着显著的挑战，尤其是在收敛缓慢和不稳定方面，这些问题因异构数据分布而加剧。在此背景下，一个关键问题是数据异构性，它增加了梯度估计的方差，并驱动模型趋向远离全局最优的局部最小值。先前的研究主要关注使用控制变量（CVs）来减少梯度估计方差而不引入偏差。在本工作中，我们提出了一种新颖的分布式梯度优化框架FedNCV，旨在有效降低梯度方差。该方法的核心是使用基于CV的REINFORCE Leave-One-Out（RLOO）技术，作为FedNCV在客户端和服务器级别的核心梯度估计器。我们基于FedNCV开发了一种算法，并提供了三项理论结果。实验评估表明，所提出的方法增强了性能。FedNCV的双结构使其能够应对联邦网络中的数据异构性和可扩展性挑战，为异构FL环境中的应用提供了一种有前景的解决方案。此外，FedNCV在Dirichlet分布下的四个不同数据集上的有效性得到了验证，与六种领先方法相比，其性能达到了新的基准。

Keyword:

control variates

federated learning

gradient optimization

期刊

Transactions on Emerging Telecommunications Technologies

IF:

2.5

论文数:

483

被引数:

3.9K

机构

Beijing University of Posts and Telecommunications

学者数:

2.6K

论文数: 1.2K

被引数: 4.2K

Southwestern University of Finance and Economics

学者数:

938

论文数: 584

被引数: 56

引用论文

Variational Inference: A Review for Statisticians 变分推断: 统计学家的评论

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION

IF3

2017-07-13

3.1K

OAAI

Blei, David M.; Kucukelbir, Alp; McAuliffe, Jon D.

Towards Optimal Customized Architecture for Heterogeneous Federated Learning With Contrastive Cloud-Edge Model Decoupling

IEEE TRANSACTIONS ON COMPUTERS

IF3.8

2024-01-01

PREAI

Chen, Xingyan; Du, Tian; Wang, Mu; Gu, Tiancheng; Zhao, Yu; Kou, Gang; Xu, Changqiao; Wu, Dapeng Oliver

Swarm Learning for decentralized and confidential clinical machine learning

Nature

IF0

2021-05-26

OAAI

Stefanie Warnat-Herresthal; Hartmut Schultze; Krishnaprasad Lingadahalli Shastry; Sathyanarayanan Manamohan; Saikat Mukherjee; Vishesh Garg; Ravi Sarveswara; Kristian Händler; Peter Pickkers; N. Ahmad Aziz; Sofia Ktena; Florian Tran; Michael Bitzer; Stephan Ossowski; Nicolas Casadei; Christian Herr; Daniel Petersheim; Uta Behrends; Fabian Kern; Tobias Fehlmann; Philipp Schommers; Clara Lehmann; Max Augustin; Jan Rybniker; Janine Altmüller; Neha Mishra; Joana P. Bernardes; Benjamin Krämer; Lorenzo Bonaguro; Jonas Schulte-Schrepping; Elena De Domenico; Christian Siever; Michael Kraut; Milind Desai; Bruno Monnet; Maria Saridaki; Charles Martin Siegel; Anna Drews; Melanie Nuesch-Germano; Heidi Theis; Jan Heyckendorf; Stefan Schreiber; Sarah Kim-Hellmuth; Paul Balfanz; Thomas Eggermann; Peter Boor; Ralf Hausmann; Hannah Kuhn; Susanne Isfort; Julia Carolin Stingl; Günther Schmalzing; Christiane K. Kuhl; Rainer Röhrig; Gernot Marx; Stefan Uhlig; Edgar Dahl; Dirk Müller-Wieland; Michael Dreher; Nikolaus Marx; Jacob Nattermann; Dirk Skowasch; Ingo Kurth; Andreas Keller; Robert Bals; Peter Nürnberg; Olaf Rieß; Philip Rosenstiel; Mihai G. Netea; Fabian Theis; Sach Mukherjee; Michael Backes; Anna C. Aschenbrenner; Thomas Ulas; Angel Angelov; Alexander Bartholomäus; Anke Becker; Daniela Bezdan; Conny Blumert; Ezio Bonifacio; Peer Bork; Bunk Boyke; Helmut Blum; Thomas Clavel; Maria Colome-Tatche; Markus Cornberg; Inti Alberto De La Rosa Velázquez; Andreas Diefenbach; Alexander Dilthey; Nicole Fischer; Konrad Förstner; Sören Franzenburg; Julia-Stefanie Frick; Gisela Gabernet; Julien Gagneur; Tina Ganzenmueller; Marie Gauder; Janina Geißert; Alexander Goesmann; Siri Göpel; Adam Grundhoff; Hajo Grundmann; Torsten Hain; Frank Hanses; Ute Hehr; André Heimbach; Marius Hoeper; Friedemann Horn; Daniel Hübschmann; Michael Hummel; Thomas Iftner; Angelika Iftner; Thomas Illig; Stefan Janssen; Jörn Kalinowski; René Kallies; Birte Kehr; Oliver T. Keppler; Christoph Klein; Michael Knop; Oliver Kohlbacher; Karl Köhrer; Jan Korbel; Peter G. Kremsner; Denise Kühnert; Markus Landthaler; Yang Li; Kerstin U. Ludwig; Oliwia Makarewicz; Manja Marz; Alice C. McHardy; Christian Mertes; Maximilian Münchhoff; Sven Nahnsen; Markus Nöthen; Francine Ntoumi; Jörg Overmann; Silke Peter; Klaus Pfeffer; Isabell Pink; Anna R. Poetsch; Ulrike Protzer; Alfred Pühler; Nikolaus Rajewsky; Markus Ralser; Kristin Reiche; Stephan Ripke; Ulisses Nunes da Rocha; Antoine-Emmanuel Saliba; Leif Erik Sander; Birgit Sawitzki; Simone Scheithauer; Philipp Schiffer; Jonathan Schmid-Burgk; Wulf Schneider; Eva-Christina Schulte; Alexander Sczyrba; Mariam L. Sharaf; Yogesh Singh; Michael Sonnabend; Oliver Stegle; Jens Stoye; Janne Vehreschild; Thirumalaisamy P. Velavan; Jörg Vogel; Sonja Volland; Max von Kleist; Andreas Walker; Jörn Walter; Dagmar Wieczorek; Sylke Winkler; John Ziebuhr; Monique M. B. Breteler; Evangelos J. Giamarellos-Bourboulis; Matthijs Kox; Matthias Becker; Sorin Cheran; Michael S. Woodacre; Eng Lim Goh; Joachim L. Schultze

A Multi-User Cost-Efficient Crowd-Assisted VR Content Delivery Solution in 5G-and-Beyond Heterogeneous Networks

IEEE TRANSACTIONS ON MOBILE COMPUTING

IF9.2

2023-08-01

PREAI

Zhong, Lujie; Chen, Xingyan; Xu, Changqiao; Ma, Yunxiao; Wang, Mu; Zhao, Yu; Muntean, Gabriel-Miro

Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data

IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING

IF7.9

2022-09-01

OAAI

Wu, Hongda; Wang, Ping

Fast-Convergent Federated Learning With Adaptive Weighting 具有自适应加权的快速收敛联邦学习

IEEE TRANSACTIONS ON COGNITIVE COMMUNICATIONS AND NETWORKING

IF7

2021-12-01

OAAI

Wu, Hongda; Wang, Ping

Advances and Open Problems in Federated Learning 联邦学习的进展和开放问题

FOUNDATIONS AND TRENDS IN MACHINE LEARNING

IF25.4

2021-01-01

2.0K

PREAI

Kairouz, Peter; McMahan, H. Brendan; Avent, Brendan; Bellet, Aurelien; Bennis, Mehdi; Bhagoji, Arjun Nitin; Bonawitz, Kallista; Charles, Zachary; Cormode, Graham; Cummings, Rachel; D'Oliveira, Rafael G. L.; Eichner, Hubert; El Rouayheb, Salim; Evans, David; Gardner, Josh; Garrett, Zachary; Gascon, Adria; Ghazi, Badih; Gibbons, Phillip B.; Gruteser, Marco; Harchaoui, Zaid; He, Chaoyang; He, Lie; Huo, Zhouyuan; Hutchinson, Ben; Hsu, Justin; Jaggi, Martin; Javidi, Tara; Joshi, Gauri; Khodak, Mikhail; Konecny, Jakub; Korolova, Aleksandra; Koushanfar, Farinaz; Koyejo, Sanmi; Lepoint, Tancrede; Liu, Yang; Mittal, Prateek; Mohri, Mehryar; Nock, Richard; Ozgur, Ayfer; Pagh, Rasmus; Qi, Hang; Ramage, Daniel; Raskar, Ramesh; Raykova, Mariana; Song, Dawn; Song, Weikang; Stich, Sebastian U.; Sun, Ziteng; Suresh, Ananda Theertha; Tramer, Florian; Vepakomma, Praneeth; Wang, Jianyu; Xiong, Li; Xu, Zheng; Yang, Qiang; Yu, Felix X.; Yu, Han; Zhao, Sen

没有更多内容