.png){kind=icon}
.png){kind=icon}
Return
Continuous Adaptive Knowledge Distillation for Few-Shot Relation Extraction
.png){kind=icon}
OA
AI
DOI：10.1145/3765749.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The goal of continuous few-shot relation extraction is to enable the model to continuously learn new relation types under conditions with limited labeled training data while avoiding the forgetting of previously learned relations. The primary challenges include catastrophic forgetting of old relations and overfitting due to data sparsity. To address these challenges, this article proposes a hierarchical distillation model that innovatively combines contrastive distillation with orthogonal adversarial distillation techniques. Specifically, we introduce a contrastive distillation approach in the feature distillation layer, integrating adaptive cosine techniques and negative sampling strategies to ensure that the model effectively retains and utilizes knowledge from previous tasks when learning new ones. Additionally, we employ orthogonal adversarial distillation in the hidden distillation layer to alleviate overfitting in low-resource scenarios. Experimental results demonstrate that our proposed method significantly outperforms the current state-of-the-art models for continuous few-shot relation extraction on two benchmark datasets, validating its effectiveness in handling few-shot data and knowledge transfer.
Keywords:
Contrastive distillation
orthogonal adversarial distillation
low-resource
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
A
IF:
0
Papers:
56
Citations:
0