Take Your Pick: Enabling Effective Distributed Learning Within Low-Dimensional Feature Space

Guogang Zhu

doi:10.1109/TNNLS.2024.3498460

Take Your Pick: Enabling Effective Distributed Learning Within Low-Dimensional Feature Space

2024-01-01

0

PRE

AI

DOI：10.1109/TNNLS.2024.3498460

Original

Original request for help

Save

Abstract

En 中文

Personalized federated learning (PFL) is a popular distributed learning framework that allows clients to have different models and has many applications where clients' data are in different domains, including autonomous driving, traffic surveillance, and medical diagnosis. The typical model of a client in PFL features a global encoder trained by all clients to extract universal features from the raw data and personalized layers (e.g., a classifier) trained using the client's local data. Nonetheless, due to the differences between the data distributions of different clients (also known as, domain gaps), the universal features produced by the global encoder largely encompass numerous components irrelevant to a certain client's local task. Some recent PFL methods address the above problem by personalizing specific parameters within the encoder. However, these methods encounter substantial challenges attributed to the high dimensionality and nonlinearity of neural network parameter space. In contrast, the feature space exhibits a lower dimensionality, providing greater intuitiveness and interpretability as compared to the parameter space. To this end, we propose a novel PFL framework named FedPick. FedPick achieves PFL within the low-dimensional feature space by adaptively selecting task-relevant features for each client from the features generated by the global encoder based on its local data distribution. It presents a more accessible and interpretable implementation of PFL compared to those methods working in the parameter space. Extensive experimental results on multiple cross-domain datasets show that FedPick can effectively select task-relevant features for each client and improve model performance in cross-domain FL.

Keywords:

Feature extraction

Adaptation models

Training

Data models

Distance learning

Computer aided instruction

Solid modeling

Distributed databases

Accuracy

Load modeling

Distributed learning

feature selection

low-dimensional feature space

personalized federated learning (PFL)

Take Your Pick: Enabling Effective Distributed Learning Within Low-Dimensional Feature Space

Abstract

Journal

Organization