.png){kind=icon}
返回
Graph Structure Enhanced Pre-Training Language Model for Knowledge Graph Completion
.png){kind=icon}
PRE
AI
DOI：10.1109/TETCI.2024.3372442.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
A vast amount of textual and structural information is required for knowledge graph construction and its downstream tasks. However, most of the current knowledge graphs are incomplete due to the difficulty of knowledge acquisition and integration. Knowledge Graph Completion (KGC) is used to predict missing connections. In previous studies, textual information and graph structural information are utilized independently, without an effective method for fusing these two types of information. In this paper, we propose a graph structure enhanced pre-training language model for knowledge graph completion. Firstly, we design a graph sampling algorithm and a Graph2Seq module for constructing sub-graphs and their corresponding contexts to support large-scale knowledge graph learning and parallel training. It is also the basis for fusing textual data and graph structure. Next, two pre-training tasks based on masked modeling are designed for capturing accurate entity-level and relation-level information. Furthermore, this paper proposes a novel asymmetric Encoder-Decoder architecture to restore masked components, where the encoder is a Pre-trained Language Model (PLM) and the decoder is a multi-relational Graph Neural Network (GNN). The purpose of the architecture is to integrate textual information effectively with graph structural information. Finally, the model is fine-tuned for KGC tasks on two widely used public datasets. The experiments show that the model achieves excellent performance and outperforms baselines in most metrics, which demonstrate the effectiveness of our approach by fusing the structure and semantic information to knowledge graph.
Keyword:
Knowledge graph completion
masked modeling
pre-trained language models
graph neural network
期刊
I
IF:
6.5
论文数:
1.5K
被引数:
5.0K
机构
引用论文
Kinematic analysis of limb movements in neuropsychological research: Subtle deficits and recovery of function.神经心理学研究中肢体运动的运动学分析: 细微的缺陷和功能的恢复。
PREAI
分享收藏KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation开普勒: 知识嵌入和预训练语言表示的统一模型
OAAI
OAAI分享收藏DBpedia - A large-scale, multilingual knowledge base extracted from WikipediaDBpedia-从维基百科中提取的大规模多语言知识库.png)
SEMANTIC WEB
OAAI
SEMANTIC WEBIF2.9
OAAI分享收藏分享收藏
查看更多内容