.png){kind=icon}
返回
Position-Aware Relational Transformer for Knowledge Graph Embedding
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2023.3262937.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Although Transformer has achieved success in language and vision tasks, its capacity for knowledge graph (KG) embedding has not been fully exploited. Using the self-attention (SA) mechanism in Transformer to model the subject-relation-object triples in KGs suffers from training inconsistency as SA is invariant to the order of input tokens. As a result, it is unable to distinguish a (real) relation triple from its shuffled (fake) variants (e.g., object-relation-subject) and, thus, fails to capture the correct semantics. To cope with this issue, we propose a novel Transformer architecture, namely, for KG embedding. It incorporates relational compositions in entity representations to explicitly inject semantics and capture the role of an entity based on its position (subject or object) in a relation triple. The relational composition for a subject (or object) entity of a relation triple refers to an operator on the relation and the object (or subject). We borrow ideas from the typical translational and semantic-matching embedding techniques to design relational compositions. We carefully design a residual block to integrate relational compositions into SA and efficiently propagate the composed relational semantics layer by layer. We formally prove that the SA with relational compositions is able to distinguish the entity roles in different positions and correctly capture relational semantics. Extensive experiments and analyses on six benchmark datasets show that achieves state-of-the-art performance on both link prediction and entity alignment.
Keyword:
Transformers
Semantics
Encoding
Training
Task analysis
Knowledge graphs
Predictive models
Entity alignment
knowledge graph (KG) embedding
link prediction
position encoding
Transformer
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
Kinematic analysis of limb movements in neuropsychological research: Subtle deficits and recovery of function.神经心理学研究中肢体运动的运动学分析: 细微的缺陷和功能的恢复。
PREAI
分享收藏
分享收藏DBpedia - A large-scale, multilingual knowledge base extracted from WikipediaDBpedia-从维基百科中提取的大规模多语言知识库.png)
SEMANTIC WEB
OAAI
SEMANTIC WEBIF2.9
OAAI分享收藏
查看更多内容