.png){kind=icon}
返回
Bayesian Recurrent Neural Network for Language Modeling
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2015.2499302.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions.
Keyword:
Bayesian learning
Hessian matrix
language model
rapid approximation
recurrent neural network
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
Chronic Pain and the Emotional Brain: Specific Brain Activity Associated with Spontaneous Fluctuations of Intensity of Chronic Back Pain慢性疼痛和情绪大脑: 与慢性背痛强度的自发性波动相关的特定大脑活动
OAAI
OAAI分享收藏Spatial mismatch analysis among hotspots of alien plant species, road and railway networks in Germany and Austria.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏
查看更多内容