.png){kind=icon}
返回
Multiuser Co-Inference With Batch Processing Capable Edge Server
.png){kind=icon}
OA
AI
DOI：10.1109/TWC.2022.3192613.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Graphics processing units (GPUs) can improve deep neural network inference throughput via batch processing, where multiple tasks are concurrently processed. We focus on novel scenarios that the energy-constrained mobile devices offload inference tasks to an edge server with GPU. The inference task is partitioned into sub-tasks for a finer granularity of offloading and scheduling, and the user energy consumption minimization problem under inference latency constraints is investigated. To deal with the coupled offloading and scheduling introduced by concurrent batch processing, we first consider an offline problem with a constant edge inference latency and the same latency constraint. It is proven that optimizing the offloading policy of each user independently and aggregating all the same sub-tasks in one batch is optimal, and thus the independent partitioning and same sub-task aggregating (IP-SSA) algorithm is inspired. Further, the optimal grouping (OG) algorithm is proposed to optimally group tasks when the latency constraints are different. Finally, when future task arrivals cannot be precisely predicted, a deep deterministic policy gradient (DDPG) agent is trained to call OG. Experiments show that IP-SSA reduces up to 94.9% user energy consumption in the offline setting, while DDPG-OG outperforms DDPG-IP-SSA by up to 8.92% in the online setting.
Keyword:
Task analysis
Servers
Batch production systems
Energy consumption
Throughput
Mobile handsets
Processor scheduling
Deep neural network (DNN) partitioning
computation offloading
batch processing
scheduling
期刊
IF:
10.7
论文数:
1.3W
被引数:
5.3W
机构
引用论文
Joint Multiuser DNN Partitioning and Computational Resource Allocation for Collaborative Edge Intelligence面向协作边缘智能的联合多用户DNN划分和计算资源分配
OAAI
OAAI分享收藏Joint Load Balancing and Offloading in Vehicular Edge Computing and Networks车载边缘计算和网络中的联合负载均衡和卸载
PREAI
分享收藏In situ genome sequencing resolves DNA sequence and structure in intact biological samples.png)
ScienceOAAI
ScienceIF0
OAAI分享收藏分享收藏分享收藏
查看更多内容