Sample-adaptive multi-branch network for efficient inference under time constraints

Lei Zhu

doi:10.1016/j.sysarc.2025.103633

Sample-adaptive multi-branch network for efficient inference under time constraints

2025-11-17

0

PRE

AI

DOI：10.1016/j.sysarc.2025.103633

Original

Original request for help

Save

Abstract

En 中文

This paper proposes a multi-branch dynamic convolutional neural network tailored for soft real-time intelligent IoT applications. The proposed network dynamically adjusts the inference path based on the hardness of input samples and time constraints of the concerned applications, and optimally trades the accuracy for latency. Specifically, we first construct the network by using a dynamic programming algorithm to automatically search for the optimal multi-branch architecture, which minimizes inference latency under the target accuracy. We then introduce a dynamic weighted training method that maximizes the accuracy of each branch. Finally, a sample-adaptive inference mechanism is developed to dynamically balance accuracy and inference latency. Experimental results show that the proposed technique improves accuracy by 0.87%, reduces computational cost by 71.45%, and achieves a 37.14% reduction in latency across multiple inference platforms. The code and models are available at: https://github.com/CheMingliang/DNet .

Sample-adaptive multi-branch network for efficient inference under time constraints

Abstract

Journal

Organization