Communication-efficient federated learning based on compressed sensing and ternary quantization

Jiali Zheng

doi:10.1007/s10489-024-05979-w

2024-12-09

PRE

Jiali Zheng *

Jing Tang (Jing Tang)

DOI：10.1007/s10489-024-05979-w

原文链接

原文求助

摘要

En 中文

Most existing work on Federated Learning (FL) transmits full-precision weights, which contain a significant amount of redundant information, leading to a substantial communication burden. This issue is particularly pronounced with the growing prevalence of smart mobile and Internet of Things (IoT) devices, where data sharing generates a large communication cost. To address this issue, we propose a communication-efficient Federated Learning algorithm, FedCSTQ, based on compressed sensing (CS) and ternary quantization.FedCSTQ introduces a heuristic sparsification method that enhances information selection, thereby mitigating the accuracy degradation typically associated with CS. Additionally, the algorithm incorporates ternary quantization to process residuals after sparsity, further reducing the impact of accuracy degradation due to sparsity while guaranteeing a small amount of communication overhead. Experiments conducted on the publicly available datasets reveal that FedCSTQ outperforms the standard FL (FedAvg), SignSGD with a majority vote, FL using dithering(CEP-FL), and FL based on Compressed Sensing (CS-FL). Ablation studies further demonstrate the effectiveness of our method.

Keyword:

Distributed learning

Collaborative work

Federated Learning (FL)

Compressed sensing (CS)

Signal processing

Quantization (signal)

Gradient compression

Communication overhead

Internet of Things (IoT)