.png){kind=icon}
返回
A Deep Multi-task Contextual Attention Framework for Multi-modal Affect Analysis
.png){kind=icon}
PRE
AI
DOI：10.1145/3380744.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-modal affect analysis (e.g., sentiment and emotion analysis) is an interdisciplinary study and has been an emerging and prominent field in Natural Language Processing and Computer Vision. The effective fusion of multiple modalities (e.g., text, acoustic, or visual frames) is a non-trivial task, as these modalities, often, carry distinct and diverse information, and do not contribute equally. The issue further escalates when these data contain noise. In this article, we study the concept of multi-task learning for multi-modal affect analysis and explore a contextual inter-modal attention framework that aims to leverage the association among the neighboring utterances and their multi-modal information. In general, sentiments and emotions have inter-dependence on each other (e.g., anger. negative or happy. positive). In our current work, we exploit the relatedness among the participating tasks in the multi-task framework. We define three different multi-task setups, each having two tasks, i.e., sentiment & emotion classification, sentiment classification & sentiment intensity prediction, and emotion classification & emotion intensity prediction. Our evaluation of the proposed system on the CMU-Multi-modal Opinion Sentiment and Emotion Intensity benchmark dataset suggests that, in comparison with the single-task learning framework, our multi-task framework yields better performance for the inter-related participating tasks. Further, comparative studies show that our proposed approach attains state-of-the-art performance for most of the cases.
Keyword:
Multi-task learning
multi-modal analysis
sentiment analysis
sentiment intensity prediction
emotion analysis
emotion intensity prediction
inter-modal attention
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.8
论文数:
1.3K
被引数:
4.4K
机构
引用论文
.png){kind=icon}
分享收藏Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages视频中的多模态情感强度分析: 面部手势和言语信息
PREAI
分享收藏
查看更多内容