.png){kind=icon}
返回
TEDT: Transformer-Based Encoding-Decoding Translation Network for Multimodal Sentiment Analysis
.png){kind=icon}
PRE
AI
DOI：10.1007/s12559-022-10073-9.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multimodal sentiment analysis is a popular and challenging research topic in natural language processing, but the impact of individual modal data in videos on sentiment analysis results can be different. In the temporal dimension, natural language sentiment is influenced by nonnatural language sentiment, which may enhance or weaken the original sentiment of the current natural language. In addition, there is a general problem of poor quality of nonnatural language features, which essentially hinders the effect of multimodal fusion. To address the above issues, we proposed a multimodal encoding-decoding translation network with a transformer and adopted a joint encoding-decoding method with text as the primary information and sound and image as the secondary information. To reduce the negative impact of nonnatural language data on natural language data, we propose a modality reinforcement cross-attention module to convert nonnatural language features into natural language features to improve their quality and better integrate multimodal features. Moreover, the dynamic filtering mechanism filters out the error information generated in the cross-modal interaction to further improve the final output. We evaluated the proposed method on two multimodal sentiment analysis benchmark datasets (MOSI and MOSEI), and the accuracy of the method was 89.3% and 85.9%, respectively. In addition, our method outperformed the current state-of-the-art methods. Our model can greatly improve the effect of multimodal fusion and more accurately analyze human sentiment.
Keyword:
Multimodal sentiment analysis
Transformer
Multimodal fusion
Multimodal attention
期刊
IF:
4.3
论文数:
1.6K
被引数:
3.6K
机构
引用论文
Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages视频中的多模态情感强度分析: 面部手势和言语信息
PREAI
分享收藏A review of affective computing: From unimodal analysis to multimodal fusion情感计算综述: 从单峰分析到多模态融合.png)
INFORMATION FUSION
OAAI
INFORMATION FUSIONIF15.5
OAAI分享收藏
分享收藏
查看更多内容