.png){kind=icon}
.png){kind=icon}
Return
EIRA: an explicit-implicit representation alignment for multimodal relation extraction
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-025-02065-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal Relation Extraction (MRE) aims to identify relations between entities by leveraging both textual and visual clues. However, prevailing MRE approaches often fall short due to two critical limitations: (1) inadequate extraction of semantic information from images, and (2) insufficiently fine-grained fusion of multimodal semantics. To bridge these gaps, we propose the Explicit-Implicit Representation and Alignment (EIRA) model. EIRA first introduces an Explicit-Implicit Representation Module that synergistically combines image captions and syntactic dependency trees to comprehensively capture image entities and their semantic relations. Subsequently, an Explicit-Implicit Feature Alignment Module is designed to achieve fine-grained semantic fusion through graph alignment and attention mechanisms, effectively aligning semantic and structural features across modalities. Finally, relation extraction is performed by integrating word embeddings with the learned fusion semantic weights. Extensive experimental results demonstrate that EIRA significantly outperforms existing state-of-the-art models, validating its effectiveness.
Keywords:
Multimodal relation extraction
Natural language processing
Image captioning
Syntactic dependency tree
Journal
IF:
3.1
Papers:
2.7K
Citations:
2.7K