.png){kind=icon}
.png){kind=icon}
Return
An energy-efficient federated learning method with gradient compression and parameter optimization
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-025-02668-z.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
To address the communication bottleneck in federated learning, in this paper, an energy-efficient gradient compression method called Qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document}Top-k-SGD, combines Top-k sparsification and stochastic quantization at s levels. A joint optimization model on FL communication cost and convergence speed is constructed to achieve the optimized compression parameters. First, the Top-k algorithm and stochastic s-level quantization are integrated into the federated learning by using sparsification to retain important gradient values and further quantizing these gradient values. A federated learning architecture based on Qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document}Top-k-SGD is proposed, and the response-based knowledge distillation technology is introduced to address the Non-i.i.d. data issue. Next, the convergence analysis of the proposed Qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document}Top-k-SGD-based federated learning algorithm is conducted, and the upper bound of the convergence speed of the proposed federated learning algorithm is quantitatively analyzed. Furthermore, by analyzing the amount of data that needs to be shared per round of iteration on devices, the communication cost of the proposed federated learning is estimated, and a joint optimization model for convergence speed and communication cost is established. With the introduction of a non-dominated sorting genetic algorithm, one non-dominated solution set for sparsification and quantization parameters is then yielded. Thereby, the Qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document}Top-k-SGD-based optimized federated learning algorithm is proposed. Finally, we validate the effectiveness of the proposed method on the MNIST, CIFAR10, and Stanford Dogs datasets. Experimental results demonstrate that the optimized Qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Q_s$$\end{document}Top-k-SGD-based federated learning shows a good balance between the communication cost and convergence speed.
Keywords:
Energy-efficient federated learning
Communication cost
Convergence speed
Gradient compression
Quantization
Sparsification
Journal
IF:
2.7
Papers:
3.1K
Citations:
5.6K