Sensitivity pruner: Filter-Level compression algorithm for deep neural networks

Suhan Guo

doi:10.1016/j.patcog.2023.109508

Sensitivity pruner: Filter-Level compression algorithm for deep neural networks

2023-08-01

9

PRE

AI

DOI：10.1016/j.patcog.2023.109508

原文链接

原文求助

摘要

En 中文

As neural networks get deeper for better performance, the demand for deployable models on resource-constrained devices also grows. In this work, we propose eliminating less sensitive filters to compress models. The previous method evaluates neuron importance using the connection matrix gradient in a single shot. To mitigate the sampling bias, we integrate this measure into the previously proposed pruning while fine-tuning framework. Besides classification errors, we introduce the difference between the learned and the single-shot strategy as the second loss component with a self-adjustive hyper-parameter that balances the training goal between improving accuracy and pruning more filters. Our Sensitivity Pruner (SP) adapts the unstructured pruning saliency metric to structured pruning tasks and enables the strategy to be derived sequentially to accommodate the updating sparsity. Experimental results demonstrate that SP significantly reduces the computational cost and the pruned models give comparable or better performance on CIFAR10, CIFAR100, and ILSVRC-12 datasets. (c) 2023 Elsevier Ltd. All rights reserved.

Keyword:

Filter pruning

Saliency-based pruning

End-to-end pruning framework

Sampling bias

Sensitivity pruner: Filter-Level compression algorithm for deep neural networks

摘要

期刊

机构

引用论文