.png){kind=icon}
返回
Sparse summary generation
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-022-03450-2.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The state-of-the-art summary generators build on powerful language models, such as BERT, which achieves impressive performance. However, most models employ softmax transformation in their output layer, leading to dense alignments and strictly positive output probabilities. This density is wasteful since it assigns probability mass to many implausible outputs. In this paper, we propose a sparse summary generation model with a new gp-entmax transformation, which includes 1.5-entmax and gradient penalty. The 1.5-entmax has the great effect of filtering noise, retaining important information and improving model performance. Experimental results show that the generated summary has improved in both ROUGE and BLEU metrics, and when tested on the CSL summarization dataset, our method outperforms the softmax model by more than 3 ROUGE-L points. For the purpose of measuring the level of important information in model-generated summaries, we propose a new metric called M2I. Simulation tests on human evaluation showed that the summary generated by the sparse model is more fluent and closer to the text's main idea.
Keyword:
Summary generation
Sparse
1
5-entmax
Softmax
期刊
IF:
3.5
论文数:
7.7K
被引数:
2.0W
机构
引用论文
分享收藏分享收藏The Early Cambrian Mianyang-Changning Intracratonic Sag and Its Control on Petroleum Accumulation in the Sichuan Basin, China.png)
Geofluids
OAAI
GeofluidsIF0
OAAI分享收藏
查看更多内容