.png){kind=icon}
返回
ARGO: Overcoming hardware dependence in distributed learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2025.107778.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Mobile devices offer a valuable resource for distributed learning alongside traditional computers, encouraging energy efficiency and privacy through local computations. However, the hardware limitations of these devices makes it impossible to use classical SGD for industry-grade machine learning models (with a very large number of parameters). Moreover, they are intermittently available and susceptible to failures. To address these challenges, we introduce ARGO, an algorithm that combines adaptive workload schemes with Byzantine resilience mechanisms, as well as dynamic device participation. Our theoretical analysis demonstrates linear convergence for strongly convex losses and sub-linear convergence for non-convex losses, without assuming specific dataset partitioning (for potential data heterogeneity). Our formal analysis highlights the interplay between convergence properties, hardware capabilities, Byzantine impact, and standard factors such as mini- batch size and learning rate. Through extensive evaluations, we show that ARGO outperforms standard SGD in terms of convergence speed and accuracy, and most importantly, thrives when classical SGD is not possible due to hardware limitations.
Keyword:
Distributed learning
Hardware heterogeneity
Byzantine resilience
Stochastic optimization
期刊
F
IF:
6.1
论文数:
6.9K
被引数:
2.3W
机构
引用论文
Total Marrow Irradiation (TMI) with Helical Tomotherapy and PBPC Following High-Dose Melphalan and PBPC as Part of Tandem Therapy for Patients with Multiple Myeloma..png)
Blood
BloodIF0
PREAI
分享收藏Adaptive Sparsification and Quantization for Enhanced Energy Efficiency in Federated Learning在联合学习中提高能量效率的自适应稀疏化和量化
OAAI
OAAI分享收藏
Two-dimensional bricklayer arrangements of tolans using halogen bonding interactions使用卤素键相互作用的tolans的二维瓦工层布置
PREAI
分享收藏
查看更多内容