.png){kind=icon}
Return
MultiJAF: Multi-modal joint entity alignment framework for multi-modal knowledge graph
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2022.05.058.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Entity Alignment (EA) is a crucial task in knowledge fusion, which aims to link entities with the same real-world identity from different Knowledge Graphs (KGs). Existing methods have achieved satisfactory performance, however, they mainly focus on single modal KG, which is difficult to be effectively applied to multi-modal scenes. In this paper, we propose a Multi-modal Joint entity Alignment Framework (MultijAF), which can effectively utilize the knowledge of various modalities. Concretely, we first learn the embeddings of different modalities, i.e., structure, attribute and image modalities. Next, we adopt an attention-based multi-modal fusion network to integrate these embeddings and use obtained joint embeddings to compute a joint embedding-based similarity matrix S-J. Moreover, we design a Numerical Process Module (NPM) to infer a similarity matrix S-N according to the numerical information of entities. In the end, we utilize a simple late fusion method to ensemble two similarity matrices for the final alignment. In addition, to reduce the cost of labeling data, we propose a novel NPM-based unsupervised multi-modal EA method. Experimental results on two real-world datasets demonstrate the effectiveness of our proposed MultijAF. (C) 2022 Elsevier B.V. All rights reserved.
Keywords:
Entity alignment
Multi-modal fusion
Multi-modal knowledge graph
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
Cited Papers
ShareSaveX-ray scattering by myosin S-1: implications for the steric blocking model of muscle control.png)
Nature
NatureIF0
PREAI
ShareSaveShareSave
Knowledge graph based natural language generation with adapted pointer-generator networksNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSaveInteractions Guided Generative Adversarial Network for unsupervised image captioningNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSaveShareSave
View more