.png){kind=icon}
返回
A systematic network pruning framework using ADMM-type algorithm
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-024-01571-x.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Regularization is a common technique to prune neural networks which removes some weights with acceptable accuracy. Moreover, the alternating direction method of multipliers (ADMM) can quickly solve regularization problems. However, for learning sparsity in deep learning, ADMM does not ensure convergence and may stop too early, since the loss is usually not convex and the stopping condition is not suitable for ADMM. In this work, we combine linearized the subproblem and iterations on mini-batches to overcome the shortcomings. To avoid ADMM converging to a local optimum, we linearize the ADMM-based subproblem. After linearizing the ADMM-based sub-problem, the objective function is convex, which can improve the convergence of ADMM theoretically. To alleviate the issue that ADMM stops too early, we increase the number of ADMM iterations by repeating the ADMM framework on mini-batches. After removing redundant parameters, the sparse model is fine-tuned in the standard way on public datasets. The experiment shows no significant accuracy reduction on CIFAR-10 when our method removes 90.55% of parameters from ResNet-20.Besides, with no drop in Top-1 accuracy on CIFAR-10, we prune 89.51% and 98.80% of the parameters for ResNet-56 and VGG-16, respectively. For ResNet-50, 73.52% of parameters are pruned without accuracy loss on ImageNet. Experimental results prove our methods outperform recent state-of-the-art pruning methods.
Keyword:
Convolutional neural networks
Alternating direction method of multipliers
Model compression
Image classification
期刊
IF:
3.1
论文数:
2.8K
被引数:
2.7K
机构
暂无机构信息
引用论文
Knowledge from the original network: restore a better pruned network with knowledge distillation
OAAI
OAAI分享收藏Calcium-modulating cyclophilin ligand regulates membrane trafficking of postsynaptic GABAA receptors
OAAI
OAAI分享收藏Students and teachers learning together: a robust training strategy for neural network pruning.png)
MULTIMEDIA SYSTEMS
MULTIMEDIA SYSTEMSIF3.1
PREAI
分享收藏GenExp: Multi-objective pruning for deep neural network based on genetic algorithmNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容