.png){kind=icon}
返回
Adaptive and Resilient Model-Distributed Inference in Edge Computing Systems
.png){kind=icon}
OA
AI
DOI：10.1109/OJCOMS.2023.3280174.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The traditional approach to distributed deep neural network (DNN) inference in edge computing systems is data-distributed inference. In this paradigm, each worker has a pre-trained DNN model. Using the DNN model, the worker processes the data that is offloaded to itself. The data-distributed inference approach (i) has high communication cost especially when the size of data is large, and (ii) is not efficient in terms of memory as the whole model should be stored and computed in each worker. Model-distributed inference is emerging as a promising solution, where a DNN model is distributed across workers. Although there is a huge amount of work on model-distributed training, the benefit of model distribution for inference is not understood well. In this paper, we analyze the potential of model-distributed inference in edge computing systems. Then, we develop an Adaptive and Resilient Model-Distributed Inference (AR-MDI) algorithm based on our optimal model allocation formulation. AR-MDI performs model allocation in a lightweight and decentralized way and it is resilient against delayed workers and failures. We implement AR-MDI in a real testbed consisting of NVIDIA Jetson TX2s and show that AR-MDI improves the inference time significantly as compared to baselines when the size of data is large, such as ImageNet.
Keyword:
& nbsp
Model-distributed inference
model parallelism
distributed deep neural networks
edge computing
期刊
I
IF:
4.3
论文数:
1.7K
被引数:
991
机构
引用论文
.png){kind=icon}
The association between chronotype profile and temporomandibular disorders among college students
PREAI
分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services
OAAI
OAAI分享收藏Distributed Learning in Wireless Networks: Recent Progress and Future Challenges无线网络中的分布式学习: 最新进展和未来挑战OAAI
OAAI分享收藏The power of obfuscation techniques in malicious JavaScript code: A measurement study恶意JavaScript代码中混淆技术的功能: 一项测量研究OAAI
OAAI分享收藏
查看更多内容