.png){kind=icon}
.png){kind=icon}
Return
Decoupled cross-attribute correlation network for multimodal sentiment analysis
.png){kind=icon}
PRE
AI
DOI：10.1016/j.inffus.2024.102897.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal sentiment analysis is a burgeoning and crucial branch in the affective computing domain with the rise of user-generated video. Most existing multimodal sentiment analysis methods focus on exploring the reinforcement of modalities through cross-modal interaction paradigms. The existing literature does not adequately explore fine-grained feature-level reinforcement, while efficient features can provide sufficient evidence for the classifier to improve model performance. To alleviate this issue, we design a novel finegrained feature-level interaction framework, Decoupled Cross-attribute Correlation Network (DCCN), to learn the multi-attributes (commonalities and diversities) of multimodal data and capture the dependencies across multi-attributes via fine-grained feature-level interactions. Specifically, we first design a decoupled multimodal representations module to learn the multi-attributes of multimodal data. Then we design a cross-attribute correlation network to capture the dependencies across multi-attributes to reinforce the features of each attribute. Finally, we design the adaptive multi-attribute decision fusion module to dynamically fuse the logits of cross-attributes and individual attributes. Extensive experiments verified the superiority of our proposed model compared with several state-of-the-art methods on two publicly available multimodal sentiment analysis benchmark datasets CMU-MOSI and CMU-MOSEI. Additionally, we can draw the following conclusions from the experiments: (i) The multimodal decoupling framework can learn the commonalities and diversity of multimodal data. (ii) The cross-attribute correlation network module can learn a higher correlation between features and labels than token-level interactions. (iii) The adaptive multi-attribute decision fusion module can dynamically adjust the contributions of multiple logits to classification.
Keywords:
Multimodal sentiment analysis
Feature reinforcement
Adaptive fusion
Journal
IF:
15.5
Papers:
4.1K
Citations:
2.7W
Organization
No organization information available