Efficient convolution-based model pruning with learn gates & BN and channel interdependence learning

doi:10.1016/j.neucom.2026.133772

Efficient convolution-based model pruning with learn gates & BN and channel interdependence learning

2026-04-30

0

PRE

AI

DOI：10.1016/j.neucom.2026.133772

Original

Original request for help

Save

Abstract

En 中文

In recent years, pruning techniques for deep neural networks have become increasingly crucial for reducing computational complexity. Traditional pruning methods often require a significant amount of time to identify network pruning structures and conduct training, thereby limiting their practical applicability. This work presents a comprehensive approach designed to efficiently handle pruning tasks while minimizing accuracy loss and reducing the overall time required for the entire pruning process. To enhance the stability and convergence capability of searching for network pruning structures, we propose Learn Gates & BN, which leverages gate parameters for each channel and existing batch normalization layers in the network to learn the importance of channels, thereby rapidly identifying pruning network structures. Furthermore, to improve the ability to accurately identify complex network pruning structures, we propose Channel Interdependence Learning (CIL), which utilizes channel attention mechanisms to learn the interdependence between channels across layers, thereby enhancing the accuracy of channel importance assessment. By leveraging channel importance assessment mechanisms in these two pruning methods, we propose Channel Importance-Based Vote Shortcut Pruning (CIVSP), which removes redundant shortcut channels to increase the pruning rate without disrupting the network’s topology. To accelerate convergence during fine-tuning after pruning, we also propose the Dynamic Loss-Driven Learning Rate Adjustment (DLLRA). This method employs a general and effective automatic learning rate adjustment strategy to adapt to the fine-tuning requirements of models of different sizes. In summary, our research provides a comprehensive approach that not only improves pruning efficiency but also achieves excellent performance in object detection and classification task.

Keywords:

pruning

channel importance

batch normalization

channel interdependence

model efficiency

Efficient convolution-based model pruning with learn gates & BN and channel interdependence learning

Abstract

Journal

Organization