.png){kind=icon}
.png){kind=icon}
Return
Multi-level inconsistency-aware fusion network for multimodal sarcasm detection
.png){kind=icon}
PRE
AI
M
L
Y
DOI：10.1007/s00530-026-02554-w.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal Sarcasm Detection (MSD) aims to uncover the sarcastic sentiments embedded within multimodal data, garnering significant attention in the field of affective computing. Although there have been notable advancements in MSD, most efforts have concentrated on integrating textual and visual information to establish cross-modal associations, often overlooking the critical role of inherent unimodal inconsistency at the textual and image levels. Moreover, existing research heavily relies on multimodal fusion features, which can lead to feature ambiguity due to conflicts or weak correlations between modalities. To address these challenges, we propose a novel multi-level inconsistency-aware fusion network. Specifically, we design a multi-level inconsistency learning module to extract multi-granular sarcastic cues from the text level, image level, and cross-modal level. To tackle the issue of cross-modal ambiguity, we introduce similarity-weighted unimodal branches that more effectively adjust the utilization of multimodal features. Finally, we incorporate a global modality-aware fusion module aimed at modeling the global relations within each modality, allowing for the interpretable integration of inconsistency features across different levels. Various experiments indicate that our proposed model can receive state-of-the-art or competitive performance.
Keywords:
Multimodal sarcasm detection
Inconsistency-aware fusion
Multi-level inconsistency learning
Cross-modal ambiguity
Modality-aware fusion
Journal
IF:
3.1
Papers:
2.7K
Citations:
2.7K
Organization
Cited Papers