Filter Pruning Based on Information Capacity and Independence

Xiaolong Tang

doi:10.1109/TNNLS.2024.3415068

Filter Pruning Based on Information Capacity and Independence

2024-01-01

0

OA

AI

DOI：10.1109/TNNLS.2024.3415068

Original

Original request for help

Save

Abstract

En 中文

Filter pruning has gained widespread adoption for the purpose of compressing and speeding up convolutional neural networks (CNNs). However, the existing approaches are still far from practical applications due to biased filter selection and heavy computation cost. This article introduces a new filter pruning method that selects filters in an interpretable, multiperspective, and lightweight manner. Specifically, we evaluate the contributions of filters from both individual and overall perspectives. For the amount of information contained in each filter, a new metric called information capacity is proposed. Inspired by the information theory, we utilize the interpretable entropy to measure the information capacity and develop a feature-guided approximation process. For correlations among filters, another metric called information independence is designed. Since the aforementioned metrics are evaluated in a simple but effective way, we can identify and prune the least important filters with less computation cost. We conduct comprehensive experiments on benchmark datasets employing various widely used CNN architectures to evaluate the performance of our method. For instance, on ILSVRC-2012, our method outperforms state-of-the-art methods by reducing floating-point operations (FLOPs) by 77.4% and parameters by 69.3% for ResNet-50 with only a minor decrease in an accuracy of 2.64%.

Keywords:

Convolutional neural network (CNN)

filter pruning

model compression

network pruning

Convolutional neural network (CNN)

filter pruning

model compression

network pruning

Filter Pruning Based on Information Capacity and Independence

Abstract

Journal

Organization