Model Compression for Communication Efficient Federated Learning

doi:10.1109/TNNLS.2021.3131614

Model Compression for Communication Efficient Federated Learning

2023-09-01

41

PRE

AI

DOI：10.1109/TNNLS.2021.3131614

Original

Original request for help

Save

Abstract

En 中文

Despite the many advantages of using deep neural networks over shallow networks in various machine learning tasks, their effectiveness is compromised in a federated learning setting due to large storage sizes and high computational resource requirements for training. A large model size can potentially require infeasible amounts of data to be transmitted between the server and clients for training. To address these issues, we investigate the traditional and novel compression techniques to construct sparse models from dense networks whose storage and bandwidth requirements are significantly lower. We do this by separately considering compression techniques for the server model to address downstream communication and the client models to address upstream communication. Both of these play a crucial role in developing and maintaining sparsity across communication cycles. We empirically demonstrate the efficacy of the proposed schemes by testing their performance on standard datasets and verify that they outperform various state-of-the-art baseline schemes in terms of accuracy and communication volume.

Keywords:

Communication efficiency

deep learning (DL)

federated learning (FL)

model compression

two-time-scale algorithms

Model Compression for Communication Efficient Federated Learning

Abstract

Journal

Organization