.png){kind=icon}
返回
DMPP: Differentiable multi-pruner and predictor for neural network pruning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2021.12.020.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Neural network pruning can trim the over-parameterized neural networks effectively by removing a number of network parameters. However, the traditional rule-based approaches always depend on manual experience. Existing heuristic search methods in discrete search spaces are usually time consuming and sub-optimal. In this paper, we develop a differentiable multi-pruner and predictor (DMPP) to prune neural networks automatically. The pruner composed of learnable parameters generates the pruning ratios of all convolutional layers as the continuous representation of the network. The neural network-based predictor is employed to predict the performance of different structures, which can accelerate the search process. Pruner and predictor enable us to directly employ gradient-based optimization to find a better structure. In addition, multi-pruner is presented to improve the efficiency of search, and knowledge distillation is leveraged to improve the performance of the pruned network. To evaluate the effectiveness of the proposed method, extensive experiments are performed on CIFAR-10, CIFAR-100, and ImageNet datasets with VGGNet and ResNet. Results show that the present DMPP can achieve a better performance than many previous state-of-the-art methods.(c) 2021 Elsevier Ltd. All rights reserved.
Keyword:
Neural network pruning
Model compression
Differentiable structure search
Performance predictor
Multi-pruner
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
.png){kind=icon}
Calcium-modulating cyclophilin ligand regulates membrane trafficking of postsynaptic GABAA receptors
OAAI
OAAI分享收藏分享收藏分享收藏
查看更多内容