.png){kind=icon}
Return
A kernel-based weight decorrelation for regularizing CNNs
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2020.11.065.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recent years has witnessed the success of convolutional neural networks (CNNs) in many machine learning and pattern recognition applications, especially in image recognition. However, due to the increasing model complexity, the parameter redundancy problem arises, and greatly degrades the performance of CNNs. To alleviate this problem, various regularization techniques, such as Dropout, have been proposed and proved their effectiveness. In this paper, we propose a novel adaptive kernel-based weight decorrelation (AKWD) framework, in order to regularize CNNs for better generalization. Different from existing works, the correlation between paring weights is measured by the cosine distance defined in RKHS associated with a specific kernel. The case with the well-known Gaussian kernel is investigated in detail, where the bandwidth parameter is adaptively estimated. By regularizing CNN models of different capacities using AKWD, better performance is achieved on several benchmark databases for both object classification and face verification tasks. In particular, when Dropout or BatchNorm is present, even higher improvements are obtained using the proposed AKWD, that demonstrates a good compatibility of the proposed regularizer with other regularization techniques. (C) 2020 Elsevier B.V. All rights reserved.
Keywords:
Convolutional neural networks
Regularization
Weight decorrelation
Object recognition
Face verification
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSave
ShareSaveShareSaveShareSave
View more