.png){kind=icon}
返回
Attention-based adaptive structured continuous sparse network pruning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.127698.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep neural network models, especially CNNs, have a wide range of applications in many fields, but their high computational power requirements limit the deployment applications in many resource-constrained embedded devices. Pruning techniques reduce the computational power requirements of models by removing redundant structures from CNNs. Most existing static pruning methods use a global uniform pruning rate to prune pretrained models and require finetuning to recover accuracy after pruning, resulting in high training costs, and the global uniform pruning rate is sub-optimal. While dynamic pruning methods perform pruning during training and use auxiliary modules to calculate the saliency scores of channels, but do not exploit its function of assisting network training. We propose an adaptive structured continuous sparse network pruning method based on the attention mechanism that prunes the original network during training. The attention -based channel similarity calculation module calculates the channel saliency scores while refining features to assist network training, and the adaptive continuous sparse control module gradually discretizes the channel saliency scores and assigns the pruning rate of each layer according to the preset pruning rate target. The pruned model is output after training and no additional fine -tuning is required. We validate the proposed method on CIFAR-10 and the large-scale dataset Imagenet using networks with different structures, and our method outperforms the comparative pruning methods at different pruning rates. In CIFAR-10, our method can reduce VGG-16 by 34.4% FLOPs while top -1 accuracy increased by 0.19%. In Imagenet, we can reduce ResNet-34 by 51.5% FLOPs, while the top -1 accuracy decreases by only 0.89%.
Keyword:
Convolutional neural networks
Model compression
Pruning
Attention
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
Caw’s Walking State Recognition Based on Accelerometers and Gyroscopes Installed on Ear-Tags and Collar-Tags基于安装在耳标和项圈上的加速度计和陀螺仪的Caw步行状态识别
.png){kind=icon}
PREAI
分享收藏
查看更多内容