Consecutive layer collaborative filter similarity for differentiable neural network pruning

doi:10.1016/j.neucom.2023.02.063

2023-05-01

PRE

Xuan Zu

Yun Li

Baoqun Yin *

DOI：10.1016/j.neucom.2023.02.063

原文链接

原文求助

摘要

En 中文

Filter pruning is proven to be an effective strategy in model compression. However, convolutional filter pruning methods usually pay all attention to evaluating filters' importance at a single layer, ignoring their collaborative relationship with corresponding filters of the next layer. In this paper, we propose novel consecutive layer collaborative filter similarity (CLCS) to make full use of the complete filter information and learn binary selection vectors to prune the redundant filters automatically. With learned selection vectors, the pruning ratio of each layer can be determined, and we can also calculate the FLOPs of the can-didate pruned network at the current stage. Under the accuracy constraint and the FLOPs constraint, the selection vectors of each layer can be optimized to achieve a better trade-off between accuracy and effi-ciency. Extensive experiments on CIFAR-10 and ImageNet with multiple networks demonstrate the effec-tiveness of our proposed method. Specifically, we obtain 54.29% and 67.33% FLOPs reduction with 0.01% and 0.09% accuracy improvement for ResNet-56 and ResNet-110 on CIFAR-10, respectively. On ImageNet, we reduce FLOPs by nearly half compared to the ResNet-50 baseline with almost no loss of accuracy. Compared with state-of-the-art filter pruning methods, our approach also achieves superior results. (c) 2023 Elsevier B.V. All rights reserved.

Keyword:

Convolutional neural network

Model compression

Network pruning

Filter similarity

期刊

Neurocomputing

IF:

6.5

论文数:

2.5W

被引数:

6.5W

机构

chinese academy of sciences

学者数:

56.7W

论文数: 45.0W

被引数: 704

引用论文

SIECP: Neural Network Channel Pruning based on Sequential Interval Estimation

NEUROCOMPUTING

IF6.5

2022-04-01

PREAI

Chen, Si-Bao; Zheng, Yu-Jie; Ding, Chris H. Q.; Luo, Bin

ImageNet Large Scale Visual Recognition Challenge ImageNet大规模视觉识别挑战

INTERNATIONAL JOURNAL OF COMPUTER VISION

IF9.3

2015-04-11

2.7W

PREAI

Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C.; Fei-Fei, Li

CCPrune: Collaborative channel pruning for learning compact convolutional networks CCPrune: 用于学习紧凑型卷积网络的协作通道修剪

NEUROCOMPUTING

IF6.5

2021-09-01

PREAI

Chen, Yanming; Wen, Xiang; Zhang, Yiwen; Shi, Weisong

Synthesis and Evaluation of Diorganotin(IV) and Triorganotin(IV) Derivatives of Aspirin, Paracetamol and Metronidazole as Antimicrobial Agents

American Journal of Drug Discovery and Development

IF0

2012-12-15