Federated Many-Task Bayesian Optimization

Hangyu Zhu

doi:10.1109/TEVC.2023.3279775

2024-08-01

PRE

Hangyu Zhu

Xilu Wang

Yaochu Jin *

DOI：10.1109/TEVC.2023.3279775

原文链接

原文求助

摘要

En 中文

Bayesian optimization (BO) is a powerful surrogate-assisted algorithm for solving expensive black-box optimization problems. While BO was developed for centralized optimization, the availability of massive distributed data has attracted increased interests in exploring federated BO (FBO) that can use data on multiple clients without leaking the raw data. However, existing FBO approaches assume that either all clients jointly solve the same optimization task, or only one client solves one target optimization task by transferring knowledge from others in a federated way, making them unsuited for many real-world applications. In this article, we consider FBO for the scenario where multiple related local black-box tasks associated with different clients are jointly optimized by sharing knowledge between tasks without leaking the data privacy. An efficient federated many-task BO framework is proposed to address not independent and identically distributed (non-IID) data while protecting the data privacy in the federated setting. A novel federated knowledge transfer paradigm is developed for dynamic many-task model aggregation according to a dissimilarity matrix. The dissimilarity is measured based on the rank of the predictions and only the hyperparameters in the local Gaussian process models are shared. In addition, a federated ensemble acquisition function is constructed by integrating the predictions of two surrogates using the global and local hyperparameters, respectively, to effectively search for the optimal solution. Experimental results show that our proposed method has reliable performance on both benchmark problems and a real machine-learning problem also in the presence of non-IID data.

Keyword:

Optimization

Bayes methods

Task analysis

Federated learning

Kernel

Distributed databases

Closed box

Bayesian optimization (BO)

federated BO (FBO)

federated learning

many-task optimization