.png){kind=icon}
返回
Accelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading
.png){kind=icon}
PRE
AI
DOI：10.1109/TPDS.2022.3222509.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
With the rapid development of Internet-of-Things (IoT) and the explosive advance of deep learning, there is an urgent need to enable deep learning inference on IoT devices in Mobile Edge Computing (MEC). To address the computation limitation of IoT devices in processing complex Deep Neural Networks (DNNs), computation offloading is proposed as a promising approach. Recently, partial computation offloading is developed to dynamically adjust task assignment strategy in different channel conditions for better performance. In this paper, we take advantage of intrinsic DNN computation characteristics and propose a novel Fused-Layer-based (FL-based) DNN model parallelism method to accelerate inference. The key idea is that a DNN layer can be converted to several smaller layers in order to increase partial computation offloading flexibility, and thus further create the better computation offloading solution. However, there is a trade-off between computation offloading flexibility as well as model parallelism overhead. Then, we investigate the optimal DNN model parallelism and the corresponding scheduling and offloading strategies in partial computation offloading. In particular, we propose a Particle Swarm Optimization with Minimizing Waiting (PSOMW) method, which explores and updates the FL strategy, path scheduling strategy, and path offloading strategy to reduce time complexity and avoid invalid solutions. Finally, we validate the effectiveness of the proposed method in commonly used DNNs. The results show that the proposed method can reduce the DNN inference time by an average of 12.75 times compared to the legacy No FL (NFL) algorithm, and is very close to the optimal solution achieved by the Brute Force (BF) algorithm with the difference of less than 0.04%.
Keyword:
Mobile edge computing
fused-layer
DNN inference
partial offloading
model parallelism
期刊
IF:
6
论文数:
5.2K
被引数:
1.1W
机构
引用论文
A Novel Mobile Edge Network Architecture with Joint Caching-Delivering and Horizontal Cooperation
OAAI
.png){kind=icon}
OAAI分享收藏Comprehensive learning particle swarm optimizer for global optimization of multimodal functions
PREAI
分享收藏Deep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge Computing面向移动边缘计算节能计算卸载的深度强化学习
PREAI
分享收藏Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks
PREAI
分享收藏
查看更多内容