Optimizing Multi-DNN Inference on Mobile Devices Through Heterogeneous Processor Co-Execution

doi:10.1109/TMC.2025.3647031

Optimizing Multi-DNN Inference on Mobile Devices Through Heterogeneous Processor Co-Execution

2025-12-23

0

PRE

AI

Chinmaya Kumar Dehury

DOI：10.1109/TMC.2025.3647031

Original

Original request for help

Save

Abstract

En 中文

Deep Neural Networks (DNNs) are increasingly adopted across various industries, driving the demand for deploying their capabilities on mobile devices. However, current mobile inference frameworks often rely on a single processor to execute each model inference, limiting hardware utilization and leading to suboptimal performance and energy efficiency. Expanding DNN accessibility on mobile platforms requires more adaptive and resource-efficient solutions to meet increasing computational demands without compromising device functionality. Nevertheless, performing parallel inference of multiple DNNs on heterogeneous processors remains a significant challenge. Existing studies have explored partitioning DNN operations into subgraphs to enable parallel execution across heterogeneous processors. However, these approaches typically generate excessive subgraphs based solely on hardware compatibility, increasing scheduling complexity and memory management overhead. To address these limitations, we propose the Advanced Multi-DNN Model Scheduling (ADMS) strategy that optimizes multi-DNN inference across heterogeneous processors on mobile devices. ADMS constructs an offline subgraph partitioning strategy that considers both hardware support for operations and scheduling granularity. It also employs a processor-state-aware scheduling algorithm to dynamically balance workloads based on real-time system conditions. This ensures efficient workload distribution and maximizes the utilization of available processors. Experimental results demonstrate that, compared to vanilla inference frameworks, ADMS achieves a 4.04× reduction in multi-DNN inference latency.

Keywords:

Deep Neural Networks

mobile devices

heterogeneous processors

co-execution

multi-DNN inference

Optimizing Multi-DNN Inference on Mobile Devices Through Heterogeneous Processor Co-Execution

Abstract

Journal

Organization