Efficient convolutional networks learning through irregular convolutional kernels

Weiyu Guo

doi:10.1016/j.neucom.2022.02.065

Efficient convolutional networks learning through irregular convolutional kernels

2022-06-01

3

OA

AI

黄永祯 (Yongzhen Huang) *

DOI：10.1016/j.neucom.2022.02.065

Original

Save

View PDF

Abstract

En 中文

As deep neural networks are increasingly used in applications suited for low-power devices, a fundamen-tal dilemma emerges: the trend is to develop models to use the increasing amount of data, resulting in memory-intensive models; however, low-power devices have very limited memory and cannot store large models. Parameters pruning is critical for deep model deployment on low-power devices. Existing efforts mainly focus on designing highly efficient structures or pruning redundant connections in networks. They are typically sensitive to the tasks or rely on dedicated and expensive hashing storage strategies. In this work, we introduce a novel approach to achieve a lightweight model from the perspec-tive of reconstructing the structure of convolution kernels for efficient storage. Our approach transforms a traditional square convolution kernel into line segments, and automatically learns a proper strategy for equipping these line segments to model diverse features. Experimental results show that our approach can significantly reduce the number of parameters (pruned 69% on DenseNet-40) and calculation costs (pruned 59% on DenseNet-40) while maintaining acceptable performance (only lose less than 2% accuracy). (c) 2022 Elsevier B.V. All rights reserved.

Keywords:

Model compression

Interpolation

Irregular convolutional kernels

Efficient convolutional networks learning through irregular convolutional kernels

Abstract

AI Summary

Journal

Organization