.png){kind=icon}
Return
A systematic network pruning framework using ADMM-type algorithm
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-024-01571-x.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Regularization is a common technique to prune neural networks which removes some weights with acceptable accuracy. Moreover, the alternating direction method of multipliers (ADMM) can quickly solve regularization problems. However, for learning sparsity in deep learning, ADMM does not ensure convergence and may stop too early, since the loss is usually not convex and the stopping condition is not suitable for ADMM. In this work, we combine linearized the subproblem and iterations on mini-batches to overcome the shortcomings. To avoid ADMM converging to a local optimum, we linearize the ADMM-based subproblem. After linearizing the ADMM-based sub-problem, the objective function is convex, which can improve the convergence of ADMM theoretically. To alleviate the issue that ADMM stops too early, we increase the number of ADMM iterations by repeating the ADMM framework on mini-batches. After removing redundant parameters, the sparse model is fine-tuned in the standard way on public datasets. The experiment shows no significant accuracy reduction on CIFAR-10 when our method removes 90.55% of parameters from ResNet-20.Besides, with no drop in Top-1 accuracy on CIFAR-10, we prune 89.51% and 98.80% of the parameters for ResNet-56 and VGG-16, respectively. For ResNet-50, 73.52% of parameters are pruned without accuracy loss on ImageNet. Experimental results prove our methods outperform recent state-of-the-art pruning methods.
Keywords:
Convolutional neural networks
Alternating direction method of multipliers
Model compression
Image classification
Journal
IF:
3.1
Papers:
2.8K
Citations:
2.7K
Organization
No organization information available
Cited Papers
Knowledge from the original network: restore a better pruned network with knowledge distillation
OAAI
OAAIShareSaveCalcium-modulating cyclophilin ligand regulates membrane trafficking of postsynaptic GABAA receptors
OAAI
OAAIShareSaveShareSaveShareSaveShareSaveStudents and teachers learning together: a robust training strategy for neural network pruning.png)
MULTIMEDIA SYSTEMS
MULTIMEDIA SYSTEMSIF3.1
PREAI
ShareSaveGenExp: Multi-objective pruning for deep neural network based on genetic algorithmNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSave
View more