.png){kind=icon}
返回
Optimizing Job Offloading Schedule for Collaborative DNN Inference
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2023.3276937.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep Neural Networks (DNNs) have been widely deployed in mobile applications. DNN inference latency is a critical metric to measure the service quality of those applications. Collaborative inference is a promising approach for latency optimization, where partial inference workloads are offloaded from mobile devices to cloud servers. Model partition problems for collaborative inference have been well studied. However, little attention has been paid to optimizing offloading pipeline for multiple DNN inference jobs. In practice, mobile devices usually need to process multiple DNN inference jobs simultaneously. We propose to jointly optimize the DNN partitioning and pipeline scheduling for multiple inference jobs. We theoretically analyze the optimal scheduling conditions for homogeneous chain-structure DNNs. Based on the analysis, we proposed near-optimal partitioning and scheduling methods for chain-structure DNNs. We also extend those methods for general-structure DNNs. In addition, we extend our problem scenario to handle heterogeneous DNN inference jobs. A layer-level scheduling algorithm is proposed. Theoretical analyses show that our proposed method is optimal when computation graphs are tree-structure. Our joint optimization methods are evaluated in a real-world testbed. Experiment results show that our methods can significantly reduce the overall inference latency of multiple inference jobs compared to partition-only or schedule-only approaches.
Keyword:
Pipelines
Processor scheduling
Mobile handsets
Task analysis
Solid modeling
Computational modeling
Cloud computing
Collaborative DNN inference
job offloading
makespan minimization
mobile cloud computing
pipeline scheduling
期刊
IF:
9.2
论文数:
5.8K
被引数:
1.8W
机构
引用论文
Abstract 13806: Catheter Ablation Improves Quasar-Arterial Spin Labeling Based Regional Brain Perfusion in the Posterior Cingulate Cortex and Auditory-verbal Memory Function in Elderly Patients With Atrial Fibrillation.png)
Circulation
CirculationIF0
PREAI
分享收藏
分享收藏Comparative evaluation of grip and pinch strength in an Asian and European populationHand Therapy
Hand TherapyIF0
PREAI
分享收藏
查看更多内容