.png){kind=icon}
.png){kind=icon}
Return
DIDS: A distributed inference framework with dynamic scheduling capability
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2024.07.037.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Distributed inference is a promising solution for deploying Deep Neural Network (DNN) applications in resource-constrained edge environments. However, due to the complexity and variability of edge scenarios, efficiently completing DNN inference remains challenging. While previous works have made significant progress in various aspects such as partition strategy, device diversity, and memory overhead, few consider the impact of environmental dynamics, particularly environmental workloads. This oversight renders existing solutions impractical. In this study, we propose DIDS, a distributed inference framework with dynamic scheduling capability that considers environmental dynamics. We first conduct a formal analysis of distributed inference and design two basic scheduling strategies: Re-Partition and Complete-Push. Then we introduce Dynamic Push, a runtime scheduler based on Complete-Push that aims to mitigate the impact of environmental workloads through runtime scheduling. The evaluation results show that DIDS can achieve up to 3.7x speedup compared to static distributed inference, in addition, DIDS is also able to achieve significant improvement compared to the state-of-the-art method.
Keywords:
Edge intelligence
Distributed computing
Runtime scheduling
AIoT
Journal
F
IF:
6.1
Papers:
6.8K
Citations:
2.3W