.png){kind=icon}
返回
A Dynamic Pruning Method on Multiple Sparse Structures in Deep Neural Networks
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3267469.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
It is well known that significant computational power and a large amount of memory are required for deep neural networks, which makes them difficult to apply in resource-limited environments. So, many network compression and acceleration technologies have emerged, of which connection pruning is widely applied due to its effectiveness and convenience. A novel connection pruning method for full model capacity on multiple sparse structures is proposed in this paper. We design a simple and efficient function called Dynamic Processing Unit (DPU) for handling the evaluated weights. Our method has the following features: 1) Instead of being pruned directly or set to 0, the weights are controlled by the DPU to determine whether they will be used during subsequent forward passes of the network during the iteration of pruning training. 2) It supports the traditional multi-steps prune method as well as the end-to-end training mode that can get a compressed network in a single stage by fusing training and pruning. 3) It can learn multiple useful sparse structures, including, but not limited to, depth-wise, filter-wise, channel-wise, 2D-filter-wise, row-wise, column-wise, connection-wise and mixed sparse structures. Our method is tested on various widely-used datasets and models, such as the LeNet and the ResNet on MNIST and CIFAR-10. Importantly, it demonstrates good performance in all these cases.
Keyword:
Convolutional neural networks
dynamic channel pruning
network compression and acceleration
structured pruning
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
查看更多内容