.png){kind=icon}
返回
Multimodal sentiment analysis based on cross-instance graph neural networks
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-05309-0.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Owing to the diversity of social media information, enhancing the accuracy of sentiment analysis for social media necessitates a comprehensive understanding of text and image information. Observing multimodal posts across social media platforms reveals similarities within images or texts containing posts, such as the association of rainbows with positive posts and darkness with negative posts. However, most previous studies have only modeled individual image-text pairs and ignored the global co-occurrence characteristics of the dataset. To address this problem, we propose a cross-instance graph neural network that leverages the global characteristics of the dataset to detect sentiments in text-image pairs. First, the method extracts five attributes from each image, constructs a co-occurrence matrix using the co-occurrence relationships between the attributes, and generates an attribute-graph convolutional network (Attribute_GCN). For the text modality, words are used as nodes, and when two words occur together more than twice, an edge is created between them. Then, point-wise mutual information and message-passing mechanisms are utilized to update the representations of the edges and nodes, resulting in the construction of Text_GNN. The hidden representations of the multimodal are obtained by encoding. Finally, a multimodal in-deep fusion with the multihead attention mechanism is implemented to better predict the sentiment of image-text pairs. We conducted extensive experiments using three public multimodal datasets, and the experimental results validated the availability of the proposed method.
Keyword:
Sentiment analysis
Multimodal
Image attribute
Model interaction
Graph convolutional network
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
暂无机构信息
引用论文
Feeding Mechanisms in Extracellular Babesia microti and Plasmodium lophurae*†细胞外 巴贝斯虫 和 疟原虫 *† 的摄食机制
PREAI
分享收藏分享收藏CMJRT: Cross-Modal Joint Representation Transformer for Multimodal Sentiment AnalysisCMJRT: 用于多模态情感分析的跨模态联合表示变换器.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏
查看更多内容