Correlation-based structural dropout for convolutional neural networks

doi:10.1016/j.patcog.2021.108117

Correlation-based structural dropout for convolutional neural networks

2021-12-01

15

PRE

AI

DOI：10.1016/j.patcog.2021.108117

Original

Original request for help

Save

Abstract

En 中文

Convolutional neural networks (CNNs) easily suffer from the over-fitting problem since they are often over-parameterized in the case of small training datasets. The conventional dropout that drops feature units randomly works well for fully connected networks, but fails to regularize CNNs well due to high spatial correlation of the intermediate features, which allows the dropped information to flow through the network, thus leading to the problem of under-dropping. To better regularize CNNs, some structural dropout methods such as SpatialDropout and DropBlock have been proposed by dropping feature units in continuous regions randomly. However, these methods may suffer from the over-dropping problem by discarding the critical discriminative features, thus limiting the performance of CNNs. To address these issues, we propose a novel structural dropout method, Correlation based Dropout (CorrDrop), to regularize CNNs by dropping feature units based on feature correlation. Unlike the previous dropout methods, our CorrDrop can focus on the discriminative information and drops features in a spatial-wise or channel wise manner. Extensive experiments on different datasets, network architectures, and various tasks (e.g., image classification and object localization) demonstrate the superiority of our method over other methods. (c) 2021 Elsevier Ltd. All rights reserved.

Keywords:

Over-fitting

Regularization

Dropout

Convolutional neural networks

Correlation-based structural dropout for convolutional neural networks

Abstract

AI Summary

Journal

Organization