.png){kind=icon}
返回
Multidimensional Dynamic Pruning: Exploring Spatial and Channel Fuzzy Sparsity
.png){kind=icon}
PRE
AI
DOI：10.1109/TFUZZ.2024.3363220.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Dynamic pruning is an effective model compression method to reduce the computational cost of networks. However, existing dynamic pruning methods are limited to pruning along a single dimension (channel, spatial, or depth), which cannot maximally excavate the redundancy of the network. Meanwhile, most of the current state-of-the-arts usually implement dynamic pruning via masked-out partial channels and pixels for training while failing to accelerate the inference speed. To tackle these limitations, we propose a novel fuzzy-based multidimensional dynamic pruning paradigm to dynamically compress neural networks along both the channel and spatial dimensions. Specifically, we design a multidimensional fuzzy-mask block to simultaneously learn which spatial positions or channels are redundant and need to be pruned. Then, the Gumbel-Softmax trick combined with a sparsity loss is introduced to train these mask modules in an end-to-end manner. During the testing stage, we convert features and convolution kernels into two matrices, respectively, and then implement sparse convolution through matrix multiplication to accelerate the network inference. Extensive experiments demonstrate that our method outperforms existing methods in terms of accuracy and computational cost. For instance, on the CIFAR-10 dataset, our method prunes 68% FLOPs of ResNet-56 with only a 0.07% Top-1 accuracy drop.
Keyword:
Computational complexity
convolutional neural networks
deep learning
image classification
neural network compression
Computational complexity
convolutional neural networks
deep learning
image classification
neural network compression
期刊
IF:
11.9
论文数:
5.0K
被引数:
2.9W
机构
引用论文
Use of Crude Extract Water–Soluble Polysaccharides of Durian (Durio zibethinus Murr) Seeds as Stabilizer for Pineapple Juice Production使用榴莲 (Durio zibethinus Murr) 种子的粗提水溶性多糖作为菠萝汁生产的稳定剂
OAAI
OAAI分享收藏Calcium-modulating cyclophilin ligand regulates membrane trafficking of postsynaptic GABAA receptorsOAAI
OAAI分享收藏分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏Dopamine-Induced Apoptosis of Lactotropes Is Mediated by the Short Isoform of D2 Receptor多巴胺诱导的催乳激素凋亡是由D2受体的短同工型介导的.png)
PLoS ONEOAAI
PLoS ONEIF0
OAAI分享收藏
查看更多内容