.png){kind=icon}
返回
Online Scheduling Algorithm for Heterogeneous Distributed Machine Learning Jobs
.png){kind=icon}
PRE
AI
DOI：10.1109/TCC.2022.3143153.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Distributed machine learning (ML) has played a key role in today's proliferation of AI services. A typical model of distributed ML is to partition training datasets over multiple worker nodes to update model parameters in parallel, adopting a parameter server or AllReduce architecture. ML training jobs are typically resource elastic, completed using various time lengths with different resource configurations. A fundamental problem in a distributed ML cluster is how to explore the demand elasticity of ML jobs and schedule them with different resource configurations, such that the utilization of resources is maximized and average job completion time is minimized. To address it, we propose an online scheduling algorithm to decide the execution time window, the number and the type of concurrent workers and parameter servers for each job upon its arrival, with a goal of minimizing the weighted average completion time. Our online algorithm consists of (i) an online scheduling framework that groups unprocessed ML training jobs into a batch iteratively, and (ii) a batch scheduling algorithm that configures each ML job to maximize the total weight of scheduled jobs in the current iteration. Our online algorithm guarantees a good parameterized competitive ratio with polynomial time complexity. Extensive evaluations using realworld data demonstrate that it outperforms state-of-the-art schedulers in today's AI cloud systems.
Keyword:
Distributed machine learning
online scheduling
期刊
I
IF:
5
论文数:
1.8K
被引数:
4.3K
机构
引用论文
A simple and nondestructive method for estimation of worker population size in Myrmica ant nests
PREAI
分享收藏
Spheroid Culture Differentially Affects Cancer Cell Sensitivity to Drugs in Melanoma and RCC Models
OAAI
OAAI分享收藏Let-7 miRNAs Modulate the Activation of NF-κB by Targeting TNFAIP3 and Are Involved in the Pathogenesis of Lupus Nephritis.png)
PLOS ONEOAAI
PLOS ONEIF0
OAAI分享收藏分享收藏
查看更多内容