.png){kind=icon}
Return
MD2I: Multi-device model-distributed neural network inference in split computing
.png){kind=icon}
PRE
AI
DOI：10.1016/j.comnet.2026.112678.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Distributed inference has emerged as a promising solution to deploy and accelerate large neural networks across resource-constrained devices. Most of the existing methods split the computation along the width dimension, and thus suffer from additional computation overhead due to overlapping input regions in Convolutional Neural Networks (CNNs) and significant communication overhead needed for aggregating global information in transformers. In addition, they require devices to store and execute entire models, thus incurring substantial memory overhead. In contrast, this paper presents MD2I, a scalable framework for multi-device model-distributed inference that partitions neural networks across the depth dimension, i.e., each device executes only a subset of layers. Unlike existing approaches, we consider dynamic wireless networking environments and the possibility of node failures. As such, we introduce layer redundancy for fault tolerance, and mathematically formulate a Hierarchical Optimal Layer Assignment and Replication Problem (HOLARP), which prioritizes normal-case performance while enabling efficient backup path when primary execution fails. We theoretically prove the NP-hardness of HOLARP and employ a two-stage Deep Reinforcement Learning (DRL) framework for real-time adaptation to changing network conditions. We have extensively evaluated MD2I on state-of-the-art neural networks including ConvNeXt and Vision Transformer (ViT) through both simulation on Colosseum with homogeneous devices and real-world testbed implementation with heterogeneous edge devices. Experimental results demonstrate that MD2I achieves up to 3.7
×
inference acceleration compared to local computing and 1.8
×
acceleration compared to existing baselines with less than 8 ms optimization overhead.
Keywords:
Distributed inference
Edge computing
Model partitioning
Latency optimization
Split computing
Journal
IF:
4.6
Papers:
1.7K
Citations:
1.6W
Organization
Cited Papers
No cited papers available