.png){kind=icon}
Return
Convolutional Neural Networks With Dynamic Regularization
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2020.2997044.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Regularization is commonly used for alleviating overfitting in machine learning. For convolutional neural networks (CNNs), regularization methods, such as DropBlock and Shake-Shake, have illustrated the improvement in the generalization performance. However, these methods lack a self-adaptive ability throughout training. That is, the regularization strength is fixed to a predefined schedule, and manual adjustments are required to adapt to various network architectures. In this article, we propose a dynamic regularization method for CNNs. Specifically, we model the regularization strength as a function of the training loss. According to the change of the training loss, our method can dynamically adjust the regularization strength in the training procedure, thereby balancing the underfitting and overfitting of CNNs. With dynamic regularization, a large-scale model is automatically regularized by the strong perturbation, and vice versa. Experimental results show that the proposed method can improve the generalization capability on off-the-shelf network architectures and outperform state-of-the-art regularization methods.
Keywords:
Training
Perturbation methods
Adaptation models
Learning systems
Convolutional neural networks
Schedules
Network architecture
Convolutional neural network (CNN)
generalization
image classification
overfitting
regularization
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
8.9
Papers:
7.6K
Citations:
7.2W
Organization
Cited Papers
Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source
.png){kind=icon}
PREAI
ShareSaveShareSaveShareSaveShareSave
View more