DoreBer: Document-Level Relation Extraction Method Based on BernNet

Boya Yuan

doi:10.1109/ACCESS.2023.3337871

2023-01-01

Boya Yuan

Liwen Xu *

DOI：10.1109/ACCESS.2023.3337871

原文链接

摘要

En 中文

Document-level relation extraction (RE) task aims to predict predefined relation types of every entity pair in a given document. Compared with the sentence-level counterpart, document-level relation extraction task requires reasoning in a more complex environment, where exist much longer text and much larger amount of entities, making it a more challenging task. However, previous methods suffers from over-smoothing problem when the count of GNN layers is high enough, leading high frequency signals on graph could not pass through filter, and then resulting in an insufficient approximation of the real function and finally causing a defective performance in tasks. To solve this problem, we propose a novel model, called DoreBer, for document-level RE task, which can obtain a higher quality of graph representation. Specifically, DoreBer performs estimation of filter over the normalized Laplacian spectrum of a graph by leveraging an order-K Bernstein polynomial approximation, and designs its spectral property by setting the coefficients of the Bernstein basis. Therefore, DoreBer can alleviate the over-smoothing problem to enhance learning ability of model. In addition, DoreBer has a higher interpretability for learned parameters of graph filter. We evaluate DoreBer on the DocRED public document-level RE dataset. Online experimental results demonstrate that DoreBer achieves significant performance improvements (2.72 and 2.76 higher on Ign F-1 and F-1 respectively), over the previous state-of-the-art on sequence-based method baseline. DoreBer reveals the potential of BernNet method in document-level relation extraction tasks and sheds light on a path to learn potential representation in high-dimensional data.The source code of this paper can be obtained from https://github.com/factor77/DoreBer/.

Keyword:

Task analysis

Feature extraction

Data mining

Cognition

Computer architecture

Recurrent neural networks

Predictive models

Natural language processing

document-level relation extraction

graph induction

graph neural network