Federated learning with clustered hyperparameter optimization

doi:10.1016/j.ins.2025.122937

Federated learning with clustered hyperparameter optimization

2025-12-01

0

PRE

AI

DOI：10.1016/j.ins.2025.122937

Original

Original request for help

Save

Abstract

En 中文

Hyperparameter tuning is crucial for improving machine learning model performance. However, the hyperparameter tuning process can be computationally expensive-typically, the entire hyper-parameter space is not searched, and some heuristic or statistically guided optimization algorithm is used. In federated learning, where there are many client models, hyperparameter tuning for each model becomes prohibitively expensive. In this work, we propose an efficient method to tune hyperparameters, by clustering clients based on their gradient updates and tuning hyperparameters individually for each cluster. We demonstrate that our proposed method leads to faster convergence compared to federated averaging, in both the i.i.d. and non-i.i.d. settings.

Keywords:

Federated learning

Bayesian optimization

Hyperparameter tuning

Clustering

Federated learning with clustered hyperparameter optimization

Abstract

Journal

Organization