.png){kind=icon}
返回
Evaluating Document Coherence Modeling
.png){kind=icon}
OA
AI
DOI：10.1162/tacl_a_00388.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
While pretrained language models (LMs) have driven impressive gains over morphosyntactic and semantic tasks, their ability to model discourse and pragmatic phenomena is less clear. As a step towards a better understanding of their discourse modeling capabilities, we propose a sentence intrusion detection task. We examine the performance of a broad range of pretrained LMs on this detection task for English. Lacking a dataset for the task, we introduce INSteD, a novel intruder sentence detection dataset, containing 170,000+ documents constructed from English Wikipedia and CNN news articles. Our experiments show that pretrained LMs perform impressively in in-domain evaluation, but experience a substantial drop in the cross-domain setting, indicating limited generalization capacity. Further results over a novel linguistic probe dataset show that there is substantial room for improvement, especially in the crossdomain setting.
期刊
T
IF:
6.9
论文数:
486
被引数:
5.7K
机构
引用论文
.png){kind=icon}
查看更多内容