.png){kind=icon}
.png){kind=icon}
Return
An efficient training-from-scratch framework with BN-based structural
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110546.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Channel pruning is an effective way of compressing convolutional neural networks (CNNs) under constrained resources. The current pruning methods follow a progressive pretrain-prune-finetune pipeline, which is inefficient and computationally expensive. In this paper, we bypass the pretrain-prune-finetune pipeline and propose a novel and efficient model training framework based on online channel pruning, which automatically produces a compact well -performed sub -network in one training -from -scratch pass under a given budget condition. Specifically, we introduce a novel BN-based indicator and a sparsity regularization strategy in the early training stage to iteratively and greedily shrink the model layers, which encourages a high -quality architecture with low channel redundancy. To ensure training stability and promote the generalization ability of the resultant pruned network, we also skillfully incorporate a simple self -distillation framework into our training and pruning pipeline. Extensive experiments indicate that our method can effectively achieve competitive performance on the image classification task compared with the state -of -the -arts.
Keywords:
Channel pruning
Model compression
Knowledge distillation
Convolutional Neural Network (CNN)
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W