.png){kind=icon}
Return
EIRA: an explicit-implicit representation alignment for multimodal relation extraction
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-025-02065-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Multimodal Relation Extraction (MRE) aims to identify relations between entities by leveraging both textual and visual clues. However, prevailing MRE approaches often fall short due to two critical limitations: (1) inadequate extraction of semantic information from images, and (2) insufficiently fine-grained fusion of multimodal semantics. To bridge these gaps, we propose the Explicit-Implicit Representation and Alignment (EIRA) model. EIRA first introduces an Explicit-Implicit Representation Module that synergistically combines image captions and syntactic dependency trees to comprehensively capture image entities and their semantic relations. Subsequently, an Explicit-Implicit Feature Alignment Module is designed to achieve fine-grained semantic fusion through graph alignment and attention mechanisms, effectively aligning semantic and structural features across modalities. Finally, relation extraction is performed by integrating word embeddings with the learned fusion semantic weights. Extensive experimental results demonstrate that EIRA significantly outperforms existing state-of-the-art models, validating its effectiveness.
Keywords:
Multimodal relation extraction
Natural language processing
Image captioning
Syntactic dependency tree
Journal
IF:
3.1
Papers:
2.8K
Citations:
2.7K
Organization
Cited Papers
BCRA: bidirectional cross-modal implicit relation reasoning and aligning for text-to-image person retrieval.png)
MULTIMEDIA SYSTEMS
MULTIMEDIA SYSTEMSIF3.1
PREAI
ShareSave
ShareSaveMFGCN: Multimodal fusion graph convolutional network for speech emotion recognitionNEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAIShareSaveDocument-level relation extraction via commonsense knowledge enhanced graph representation learningAPPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
ShareSaveTSPNet: Translation supervised prototype network via residual learning for multimodal social relation extractionNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSaveEncoder-decoder with bilateral gated fusion for multimodal relation extractionMULTIMEDIA SYSTEMS
MULTIMEDIA SYSTEMSIF3.1
PREAI
ShareSaveShareSave
View more