.png){kind=icon}
返回
Language Model Guided Knowledge Graph Embeddings
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2022.3191666.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Knowledge graph embedding models have become a popular approach for knowledge graph completion through predicting the plausibility of (potential) triples. This is performed by transforming the entities and relations of the knowledge graph into an embedding space. However, knowledge graphs often include further textual information stored in literal, which is ignored by such embedding models. As a consequence, the learning process stays limited to the structure and the connections between the entities, which has the potential to negatively influence the performance. We bridge this gap by leveraging the capabilities of pre-trained language models to include textual knowledge in the learning process of embedding models. This is achieved by introducing a new loss function that guides embedding models in measuring the likelihood of triples by taking such complementary knowledge into consideration. The proposed solution is a model-independent loss function that can be plugged into any knowledge graph embedding model. In this paper, Sentence-BERT and fastText are used as pre-trained language models from which the embeddings of the textual knowledge are obtained and injected into the loss function. The loss function contains a trainable slack variable that determines the degree to which the language models influence the plausibility of triples. Our experimental evaluation on six benchmarks, namely Nations, UMLS, WordNet, and three versions of CodEx confirms the advantage of using pre-trained language models for boosting the accuracy of knowledge graph embedding models. We showcase this by performing evaluations on top of the five well-known knowledge graph embedding models such as TransE, RotatE, ComplEx, DistMult, and QuatE. The results show an improvement in accuracy up to 9% on UMLS dataset for the Distmult model and 4.2% on the Nations dataset for the ComplEx model when they are guided by pre-trained language models. We additionally studied the effect of multiple factors such as the structure of the knowledge graphs and training steps and presented them as ablation studies.
Keyword:
Predictive models
Computational modeling
Task analysis
Electronic mail
Adaptation models
Unified modeling language
Licenses
Knowledge graph
knowledge graph embeddings
language models
link prediction
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Ligand substitution reactions of metal and organometal carbonyls with group V and VI donor ligands
PREAI
分享收藏Exceptionally preserved soft parts in fossils from the Lower Ordovician of Morocco clarify stylophoran affinities within basal deuterostomes.png)
Geobios
OAAI
GeobiosIF0
OAAI分享收藏KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation开普勒: 知识嵌入和预训练语言表示的统一模型OAAI
OAAI分享收藏分享收藏Fault geometry and permeability contrast control vent temperatures at the Logatchev 1 hydrothermal field, Mid-Atlantic RidgeGeology
GeologyIF0
PREAI
分享收藏Cobalt and Copper Composite Oxides as Efficient Catalysts for Preferential Oxidation of CO in H2-Rich Stream钴和铜复合氧化物作为H2-Rich流中CO优先氧化的有效催化剂
PREAI
分享收藏
查看更多内容