.png){kind=icon}
返回
Sequence-aware Knowledge Distillation for a Lightweight Event Representation
.png){kind=icon}
PRE
AI
DOI：10.1145/3545798.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Event representation targets to model the event-reasoning process as a machine-readable format. Previous studies on event representation mostly concentrate on a sole modeling perspective and have not well investigated the scenario-level knowledge, which can cause information loss. To cope with this dilemma, we propose a unified fine-tuning architecture-based approach (UniFA-S) that integrates all levels of trainings, including the scenario-level knowledge. However, another challenge for existing models is the ever-increasing computation overheads, restricting the deployment ability on limited resources devices. Hence, in this article, we aim to compress the cumbersome model UniFA-S into a lighter and easy-to-deploy one without much performance damage. To this end, we propose a sequence-aware knowledge distillation model (SaKD) that employs a dynamic self-distillation on the decouple-compress-couple framework for compressing UniFA-S, which cannot only realize the model compression, but also retain the integrity of individual components. We also design two fitting strategies to address the less-supervised issue at the distillation stage. Comprehensive experiments on representation-and-inference ability-based tasks validate the effectiveness of SaKD. Compared to UniFA-S, SaKD realizes a more portable event representation model at the cost of only 1.0% performance drop in terms of accuracy or Spearman's correlation, which is far less than other knowledge distillation models.
Keyword:
Event representation
knowledge distillation
event similarity
event prediction
期刊
IF:
9.1
论文数:
1.2K
被引数:
4.7K
机构
引用论文
.png){kind=icon}
Pre-train, Interact, Fine-tune: a novel interaction representation for text classification预训练，交互，微调: 一种新颖的文本分类交互表示
OAAI
OAAI分享收藏Synthesis of a new fluorelastomer from tetrafluoroethylene and propylene by radiation processing
PREAI
分享收藏
查看更多内容