.png){kind=icon}
Return
Multi-Exit DNN Inference Acceleration Based on Multi-Dimensional Optimization for Edge Intelligence
.png){kind=icon}
PRE
AI
DOI：10.1109/TMC.2022.3172402.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Edge intelligence, as a prospective paradigm for accelerating DNN inference, is mostly implemented by model partitioning which inevitably incurs the large transmission overhead of DNN's intermediate data. A popular solution introduces multi-exit DNNs to reduce latency by enabling early exits. However, existing work ignores the correlation between exit settings and synergistic inference, causing incoordination of device-to-edge. To address this issue, this paper first investigates the bottlenecks of executing multi-exit DNNs in edge computing and builds a novel model for inference acceleration with exit selection, model partition, and resource allocation. To tackle the intractable coupling subproblems, we propose a Multi-exit DNN inference Acceleration framework based on Multi-dimensional Optimization (MAMO). In MAMO, the exit selection subproblem is first extracted from the original problem. Then, bidirectional dynamic programming is employed to determine the optimal exit setting for an arbitrary multi-exit DNN. Finally, based on the optimal exit setting, a DRL-based policy is developed to learn joint decisions of model partition and resource allocation. We deploy MAMO on a real-world testbed and evaluate its performance in various scenarios. Extensive experiments show that it can adapt to heterogeneous tasks and dynamic networks, and accelerate DNN inference by up to 13:7x compared with the state-of-the-art.
Keywords:
Edge intelligence
exit selection
inference acceleration
model partition
multi-exit DNN
resource allocation
Journal
IF:
9.2
Papers:
5.9K
Citations:
1.8W
Organization
View more organizations
Cited Papers
ShareSaveEdge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing.png)
PROCEEDINGS OF THE IEEE
OAAI
PROCEEDINGS OF THE IEEEIF25.9
OAAIShareSaveShareSaveShareSaveShareSaveIn-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated LearningIEEE NETWORKOAAI
IEEE NETWORKIF6.3
OAAIShareSaveShareSave
View more