ESCAN: Efficient GPU sharing for cascade neural network inference

Wang, Jianan

doi:10.1016/j.neunet.2025.107703

ESCAN: Efficient GPU sharing for cascade neural network inference

2025-06-15

0

PRE

AI

DOI：10.1016/j.neunet.2025.107703

Original

Original request for help

Save

Abstract

En 中文

• We address the deployment challenges of cascaded models in GPU-sharing scenarios. • GPU sharing requires balancing parallel gains and redundant computation for cascade models. • We introduce ESCAN with batch-parallel execution and resource allocation. • ESCAN improves the efficiency of cascaded models by optimizing the execution mode and resource allocation. • ESCAN simplifies and enhances the efficiency of the optimization process.

ESCAN: Efficient GPU sharing for cascade neural network inference

Abstract

Journal

Organization