.png){kind=icon}
返回
Pre-training the deep generative models with adaptive hyperparameter optimization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2017.03.058.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The performance of many machine learning algorithms depends crucially on the hyperparameter settings, especially in Deep Learning. Manually tuning the hyperparameters is laborious and time consuming. To address this issue, Bayesian optimization (BO) methods and their extensions have been proposed to optimize the hyperparameters automatically. However, they still suffer from highly computational expense when applying to deep generative models (DGMs) due to their strategy of the black-box function optimization. This paper provides a new hyperparameter optimization procedure at the pre-training phase of the DGMs, where we avoid combining all layers as one black-box function by taking advantage of the layer-by-layer learning strategy. Following this procedure, we are able to optimize multiple hyperparameters in an adaptive way by using Gaussian process. In contrast to the traditional BO methods, which mainly focus on the supervised models, the pre-training procedure is unsupervised where there is no validation error can be used. To alleviate this problem, this paper proposes a new holdout loss, the free energy gap, which takes into account both factors of the model fitting and over-fitting. The empirical evaluations demonstrate that our method not only speeds up the process of hyperparameter optimization, but also improves the performances of DGMs significantly in both the supervised and unsupervised learning tasks. (C) 2017 Elsevier B.V. All rights reserved.
Keyword:
Deep generative model
Hyperparameter optimization
Sequential model-based optimization
Contrastive divergence
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.5
论文数:
2.5W
被引数:
7.5W
机构
引用论文
分享收藏分享收藏Unsupervised learning by probabilistic latent semantic analysis基于概率潜在语义分析的无监督学习.png)
MACHINE LEARNING
OAAI
MACHINE LEARNINGIF2.9
OAAI分享收藏Time series forecasting using a deep belief network with restricted Boltzmann machines使用具有受限玻尔兹曼机的深度信念网络进行时间序列预测NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容