.png){kind=icon}
返回
Encoding-based memory for recurrent neural networks
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.04.051.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Learning to solve sequential tasks with recurrent models requires the ability to memorize long sequences and to extract task-relevant features from them. In this paper, we study memorization from the point of view of the design and training of recurrent neural networks. We study how to maximize the short-term memory of recurrent units, an objective difficult to achieve using backpropagation. We propose a new model, the Linear Memory Network, which features an encoding-based memorization component built with a linear autoencoder for sequences. Additionally, we provide a specialized training algorithm that initializes the memory to efficiently encode the hidden activations of the network. Experimental results on synthetic and real-world datasets show that the chosen encoding mechanism is superior to static encodings such as orthogonal models and the delay line. The method also outperforms RNN and LSTM units trained using stochastic gradient descent. Experiments on symbolic music modeling show that the training algorithm specialized for the memorization component improves the final performance compared to stochastic gradient descent. (c) 2021 Elsevier B.V. All rights reserved.
Keyword:
Recurrent neural networks
Autoencoders
Linear dynamical systems
Memory capacity
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
A review of unsupervised feature learning and deep learning for time-series modeling用于时间序列建模的无监督特征学习和深度学习综述
OAAI
OAAI分享收藏分享收藏Effect of MoS2 and (phosphorene, germanene, borophene) hybrid structure on the performance of an SPR biosensor for detection of bacteria二硫化钼（MoS2）与（磷烯、锗烯、硼烯）杂化结构对细菌检测表面等离子体共振（SPR）生物传感器性能的影响
PREAI
分享收藏Borophene Quantum Dots with Strong Photoluminescence for Selective Metal Ion Sensing具有强光致发光性的硼烯量子点，用于选择性金属离子传感
PREAI
分享收藏
查看更多内容