.png){kind=icon}
.png){kind=icon}
Return
Progressive kernel pruning CNN compression method with an adjustable input channel
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-021-02932-z.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural network pruning is an effective model compression and acceleration method. In the initial pruning stage, maintaining the integrity of the input channel of the convolution layer is very important to improve the performance of the pruning model. This paper proposes a two-stage multi-strategy progressive kernel pruning method with adjustable input channels. First, a Hybrid Norm Sparse Index (HNSI) is defined as the basis for selecting the number of retained kernels, and then a two-stage progressive pruning technique is adopted. In the first stage, HNSI is used in the groups for moderate kernel pruning. HNSI in the group can reserve at least one kernel in each group, allowing the input channel information of each layer to be mapped to the next layer and ensuring better network optimization through moderate pruning. In the second stage, HNSI is used in the layers for adjustable full kernel pruning. At this stage, the HNSI of each layer is the basis of preserving the kernel number, and the pruning process is divided into two strategies. The first strategy is kernel pruning in the low-level layer. On the basis of the overall kernel pruning in the layer, each group is forced to retain at least one kernel, thus ensuring that the primary feature of each input channel can be transmitted to the next layer. The second strategy is kernel pruning in the high-level layer. Because of the stronger information abstraction ability in the high-level layer, only the valid input channel information can be passed to the next layer, no longer forcing each group to retain the kernel, which can greatly improve the efficiency of network pruning. Model analysis and experiments show that the two-stage kernel pruning can not only obtain better network optimization direction under moderate pruning but also obtain better network performance under a higher pruning rate.
Keywords:
Convolutional neural network
Progressive kernel pruning
Hybrid norm sparse index
Compression
Acceleration