.png){kind=icon}
返回
The Fast Inertial ADMM optimization framework for distributed machine learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2024.107575.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The ADMM (Alternating Direction Method of Multipliers) optimization framework is known for its property of decomposition and assembly, which effectively bridges distributed computing and optimization algorithms, making it well-suited for distributed machine learning in the context of big data. However, it suffers from slow convergence speed and lacks the ability to coordinate worker computations, resulting in inconsistent speeds in solving subproblems in distributed systems and mutual waiting among workers. In this paper, we propose a novel optimization framework to address these challenges in support vector regression (SVR) and probit regression training through the FIADMM (Fast I nertial ADMM). The key concept of the FIADMM lies in the introduction of inertia acceleration and an adaptive subproblem iteration mechanism based on the ADMM, aimed at accelerating convergence speed and reducing the variance in solving speeds among workers. Further, we prove that FIADMM has a fast linear convergence rate O (1/k). Experimental results on six benchmark datasets demonstrate that the proposed FIADMM significantly enhances convergence speed and computational efficiency compared to multiple baseline algorithms and related efforts.
Keyword:
ADMM
Optimization framework
Distributed machine learning
Inertial acceleration
期刊
F
IF:
6.1
论文数:
6.8K
被引数:
2.3W
机构
引用论文
Accelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading通过模型并行性和部分计算卸载加速深度学习推理
PREAI
分享收藏A Method of Inertial Regularized ADMM for Separable Nonconvex Optimization Problems.png)
SOFT COMPUTING
SOFT COMPUTINGIF2.5
PREAI
分享收藏
查看更多内容