.png){kind=icon}
返回
Topic-aware hierarchical multi-attention network for text classification
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-022-01734-0.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Neural networks, primarily recurrent and convolutional Neural networks, have been proven successful in text classification. However, convolutional models could be limited when classification tasks are determined by long-range semantic dependency. While the recurrent ones can capture long-range dependency, the sequential architecture of which could constrain the training speed. Meanwhile, traditional networks encode the entire document in a single pass, which omits the hierarchical structure of the document. To address the above issues, this study presents T-HMAN, a Topic-aware Hierarchical Multiple Attention Network for text classification. A multi-head self-attention coupled with convolutional filters is developed to capture long-range dependency via integrating the convolution features from each attention head. Meanwhile, T-HMAN combines topic distributions generated by Latent Dirichlet Allocation (LDA) with sentence-level and document-level inputs respectively in a hierarchical architecture. The proposed model surpasses the accuracies of the current state-of-the-art hierarchical models on five publicly accessible datasets. The ablation study demonstrates that the involvement of multiple attention mechanisms brings significant improvement. The current topic distributions are fixed vectors generated by LDA, the topic distributions will be parameterized and updated simultaneously with the model weights in future work.
Keyword:
Text classification
Topic model
Attention mechanism
Natural language processing
期刊
IF:
2.7
论文数:
3.2K
被引数:
5.6K
机构
暂无机构信息
引用论文
Structural neural correlates of prosaccade and antisaccade eye movements in healthy humans.png)
NeuroImage
NeuroImageIF0
PREAI
分享收藏Regional expression of MTG genes in the developing mouse central nervous systemMTG基因在发育中的小鼠中枢神经系统中的区域表达
OAAI
OAAI分享收藏分享收藏
查看更多内容