A Multi-Modal Entity Alignment Method with Inter-Modal Enhancement

Song Yuan

doi:10.3390/bdcc7020077

A Multi-Modal Entity Alignment Method with Inter-Modal Enhancement

2023-04-18

2

OA

AI

DOI：10.3390/bdcc7020077

Original

Original request for help

Save

Abstract

En 中文

Due to inter-modal effects hidden in multi-modalities and the impact of weak modalities on multi-modal entity alignment, a Multi-modal Entity Alignment Method with Inter-modal Enhancement (MEAIE) is proposed. This method introduces a unique modality called numerical modality in the modal aspect and applies a numerical feature encoder to encode it. In the feature embedding stage, this paper utilizes visual features to enhance entity relation representation and influence entity attribute weight distribution. Then, this paper introduces attention layers and contrastive learning to strengthen inter-modal effects and mitigate the impact of weak modalities. In order to evaluate the performance of the proposed method, experiments are conducted on three public datasets: FB15K, DB15K, and YG15K. By combining the datasets in pairs, compared with the current state-of-the-art multi-modal entity alignment models, the proposed model achieves a 2% and 3% improvement in Top-1 Hit Rate(Hit@1) and Mean Reciprocal Rank (MRR), demonstrating its feasibility and effectiveness.

Keywords:

multi-modal entity alignment

feature encoding

inter-modal effects

contrastive learning

attention

A Multi-Modal Entity Alignment Method with Inter-Modal Enhancement

Abstract

Journal

Organization