.png){kind=icon}
返回
Virtual prompt pre-training for prototype-based few-shot relation extraction
.png){kind=icon}
OA
AI
DOI：10.1016/j.eswa.2022.118927.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Prompt tuning with pre-trained language models (PLM) has exhibited outstanding performance by reducing the gap between pre-training tasks and various downstream applications, which requires additional labor efforts in label word mappings and prompt template engineering. However, in a label intensive research domain, e.g., few-shot relation extraction (RE), manually defining label word mappings is particularly challenging, because the number of utilized relation label classes with complex relation names can be extremely large. Besides, the manual prompt development in natural language is subjective to individuals. To tackle these issues, we propose a virtual prompt pre-training method, projecting the virtual prompt to latent space, then fusing with PLM parameters. The pre-training is entity-relation-aware for RE, including the tasks of mask entity prediction, entity typing, distant supervised RE, and contrastive prompt pre-training. The proposed pre-training method can provide robust initialization for prompt encoding, while maintaining the interaction with the PLM. Furthermore, the virtual prompt can effectively avoid the labor efforts and the subjectivity issue in label word mapping and prompt template engineering. Our proposed prompt-based prototype network delivers a novel learning paradigm to model entities and relations via the probability distribution and Euclidean distance of the predictions of query instances and prototypes. The results indicate that our model yields an averaged accuracy gain of 4.21% on two few-shot datasets over strong RE baselines. Based on our proposed framework, our pre-trained model outperforms the strongest RE-related PLM by 6.52%.
Keyword:
Few-shot learning
Information extraction
Prompt tuning
Pre-trained Language Model
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
引用论文
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation开普勒: 知识嵌入和预训练语言表示的统一模型
OAAI
OAAI分享收藏MetaPro: A computational metaphor processing model for text pre-processing.png)
INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
分享收藏Database resources of the national center for biotechnology information国家生物技术信息中心数据库资源NUCLEIC ACIDS RESEARCHOAAI
NUCLEIC ACIDS RESEARCHIF13.1
OAAI分享收藏分享收藏Construction of Genealogical Knowledge Graphs From Obituaries: Multitask Neural Network Extraction System从讣告构建系谱知识图谱: 多任务神经网络提取系统OAAI
OAAI分享收藏 没有更多内容
没有更多内容