.png){kind=icon}
返回
Surrogate dropout: Learning optimal drop rate through proxy
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2020.106340.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Dropout is commonly used in deep neural networks to alleviate the problem of overfitting. Conventionally, the neurons in a layer indiscriminately share a fixed drop probability, which results in difficulty in determining the appropriate value for different tasks. Moreover, this static strategy will also incur serious degradation on performance when the conventional dropout is extensively applied to both shallow and deep layers. A question is whether selectively dropping the neurons would realize a better regularization effect. This paper proposes a simple and effective surrogate dropout method whereby neurons are dropped according to their importance. The proposed method has two main stages. The first stage trains a surrogate module that can be jointly optimized along with the neural network to evaluate the importance of each neuron. In the second stage, the output of the surrogate module is regarded as a guidance signal for dropping certain neurons, approximating the optimal per neuron drop rate when the network converges. Various convolutional neural network architectures and multiple datasets, including CIFAR-10, CIFAR-100, SVHN, Tiny ImageNet, and two medical image datasets are used to evaluate the surrogate dropout method. The experimental results demonstrate that the proposed method achieves a better regularization effect than the baseline methods. (C) 2020 Elsevier B.V. All rights reserved.
Keyword:
Deep neural networks
Dropout
Regularization
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏Automated segmentation of macular edema in OCT using deep neural networks.png)
MEDICAL IMAGE ANALYSIS
MEDICAL IMAGE ANALYSISIF11.8
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
分享收藏
查看更多内容