Communication-Efficient Federated Learning via Predictive Coding

Kai Yue

doi:10.1109/JSTSP.2022.3142678

Communication-Efficient Federated Learning via Predictive Coding

2022-04-01

7

OA

AI

DOI：10.1109/JSTSP.2022.3142678

Original

Original request for help

Save

Abstract

En 中文

Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical bottleneck due to limited power and bandwidth. Prior work has utilized various data compression tools such as quantization and sparsification to reduce the overhead. In this paper, we propose a predictive coding based compression scheme for federated learning. The scheme has shared prediction functions among all devices and allows each worker to transmit a compressed residual vector derived from the reference. In each communication round, we select the predictor and quantizer based on the rate-distortion cost, and further reduce the redundancy with entropy coding. Extensive simulations reveal that the communication cost can be reduced up to 99% with even better learning performance when compared with other baseline methods.

Keywords:

Predictive models

Servers

Collaborative work

Predictive coding

Entropy coding

Costs

Quantization (signal)

Federated learning

distributed optimization

predictive coding

Communication-Efficient Federated Learning via Predictive Coding

Abstract

Journal

Organization