.png){kind=icon}
返回
Attention in Natural Language Processing
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2020.3019893.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain.
Keyword:
Task analysis
Computer architecture
Visualization
Neural networks
Natural language processing
Taxonomy
Computational modeling
Natural language processing (NLP)
neural attention
neural networks
review
survey
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
分享收藏分享收藏Structural neural correlates of prosaccade and antisaccade eye movements in healthy humans.png)
NeuroImage
NeuroImageIF0
PREAI
分享收藏分享收藏
查看更多内容