.png){kind=icon}
.png){kind=icon}
Return
A Dynamic Pruning Method on Multiple Sparse Structures in Deep Neural Networks
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3267469.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
It is well known that significant computational power and a large amount of memory are required for deep neural networks, which makes them difficult to apply in resource-limited environments. So, many network compression and acceleration technologies have emerged, of which connection pruning is widely applied due to its effectiveness and convenience. A novel connection pruning method for full model capacity on multiple sparse structures is proposed in this paper. We design a simple and efficient function called Dynamic Processing Unit (DPU) for handling the evaluated weights. Our method has the following features: 1) Instead of being pruned directly or set to 0, the weights are controlled by the DPU to determine whether they will be used during subsequent forward passes of the network during the iteration of pruning training. 2) It supports the traditional multi-steps prune method as well as the end-to-end training mode that can get a compressed network in a single stage by fusing training and pruning. 3) It can learn multiple useful sparse structures, including, but not limited to, depth-wise, filter-wise, channel-wise, 2D-filter-wise, row-wise, column-wise, connection-wise and mixed sparse structures. Our method is tested on various widely-used datasets and models, such as the LeNet and the ResNet on MNIST and CIFAR-10. Importantly, it demonstrates good performance in all these cases.
Keywords:
Convolutional neural networks
dynamic channel pruning
network compression and acceleration
structured pruning