Fedpower: privacy-preserving distributed eigenspace estimation

Xiao Guo

doi:10.1007/s10994-024-06620-0

Fedpower: privacy-preserving distributed eigenspace estimation

2024-09-26

0

PRE

AI

DOI：10.1007/s10994-024-06620-0

Original

Original request for help

Save

Abstract

En 中文

Eigenspace estimation is a fundamental tool in data analytics, which has found applications in PCA, dimension reduction, and clustering, among others. The modern machine learning community usually involves data that come from and belong to different organizations. The low communication power and possible data privacy breaches make the eigenspace estimation challenging. To address these issues, we propose a class of algorithms called FedPower within the federated learning (FL) framework. FedPower leverages the well-known power method by alternating multiple local power iterations and a global aggregation step, thus improving communication efficiency. In the aggregation, we propose to weight each local eigenvector matrix with Orthogonal Procrustes Transformation (OPT) for better alignment. We add Gaussian noise in each iteration to ensure strong privacy protection by adopting the notion of differential privacy (DP). We provide convergence bounds for FedPower composed of different interpretable terms corresponding to the effects of Gaussian noise, parallelization, and random sampling of local machines. Additionally, we conduct experiments to demonstrate the effectiveness of our proposed algorithms.

Keywords:

Communication efficiency

Federated learning

Power method

Stragglers' effect

Fedpower: privacy-preserving distributed eigenspace estimation

Abstract

Journal

Organization