.png){kind=icon}
返回
Maximum Relevance Minimum Redundancy Dropout with Informative Kernel Determinantal Point Process
.png){kind=icon}
OA
AI
DOI：10.3390/s21051846.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In recent years, deep neural networks have shown significant progress in computer vision due to their large generalization capacity; however, the overfitting problem ubiquitously threatens the learning process of these highly nonlinear architectures. Dropout is a recent solution to mitigate overfitting that has witnessed significant success in various classification applications. Recently, many efforts have been made to improve the Standard dropout using an unsupervised merit-based semantic selection of neurons in the latent space. However, these studies do not consider the task-relevant information quality and quantity and the diversity of the latent kernels. To solve the challenge of dropping less informative neurons in deep learning, we propose an efficient end-to-end dropout algorithm that selects the most informative neurons with the highest correlation with the target output considering the sparsity in its selection procedure. First, to promote activation diversity, we devise an approach to select the most diverse set of neurons by making use of determinantal point process (DPP) sampling. Furthermore, to incorporate task specificity into deep latent features, a mutual information (MI)-based merit function is developed. Leveraging the proposed MI with DPP sampling, we introduce the novel DPPMI dropout that adaptively adjusts the retention rate of neurons based on their contribution to the neural network task. Empirical studies on real-world classification benchmarks including, MNIST, SVHN, CIFAR10, CIFAR100, demonstrate the superiority of our proposed method over recent state-of-the-art dropout algorithms in the literature.
Keyword:
deep learning
regularization methods
dropout
determinantal point process
information theory
image classification
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.5
论文数:
7.2W
被引数:
20.9W
机构
引用论文
Spatio-Temporal Graph Deep Neural Network for Short-Term Wind Speed Forecasting用于短期风速预测的时空图深度神经网络
PREAI
分享收藏Dropout vs. batch normalization: an empirical study of their impact to deep learning辍学与批量归一化: 对深度学习影响的实证研究
PREAI
分享收藏Mutual information-based dropout: Learning deep relevant feature representation architectures.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
查看更多内容