.png){kind=icon}
返回
Multimodal hypergraph network with contrastive learning for sentiment analysis
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.129566.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multimodal Sentiment Analysis (MSA) is the process of relying on multimodal information, such as text, audio, and visual, to determine a subject's affective tendencies. While many recent studies have adopted graph-based techniques for MSA, they have yet to fully explore the sentimental interactions both within unimodal temporal steps and across individual modalities. To address the limitations arising from the isolation of unimodal hypergraphs in affective relationship mining, this paper proposes a multimodal hypergraph network based on contrastive learning. It constructs the hypergraph structure by utilizing the sequential time steps of all three modalities as a collection of nodes, aiming to explore the multidimensional affective relationships across uni-, bi-, and tri-modalities. Specifically, this paper first generates the initial hypergraph structure using a correlation-based hypergraph construction method to ensure the effectiveness of the constructed hypergraph. Then, both supervised and unsupervised contrastive learning methods are designed to optimize feature learning and the structure of the multimodal hypergraph, adaptively and simultaneously capturing the relationships among the time-series nodes of uni-, bi-, and tri-modalities. The proposed methods in this paper have demonstrated the advantages and effectiveness by conducting a large number of comparative and validation experiments on the English CMU-MOSI and CMU-MOSEI datasets as well as the Chinese CH-SIMS dataset.
Keyword:
Multimodal sentiment analysis
Hypergraph structure optimization
Contrastive learning
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages视频中的多模态情感强度分析: 面部手势和言语信息
PREAI
分享收藏Dynamic hypergraph convolutional network for multimodal sentiment analysis用于多模态情感分析的动态超图卷积网络.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容