.png){kind=icon}
返回
A Two-Stage Transformer-Based Approach for Variable-Length Abstractive Summarization
.png){kind=icon}
PRE
AI
DOI：10.1109/TASLP.2020.3006731.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This study proposes a two-stage method for variable-length abstractive summarization. This is an improvement over previous models, in that the proposed approach can simultaneously achieve fluent and variable-length abstractive summarization. The proposed abstractive summarization model consists of a text segmentation module and a two-stage Transformer-based summarization module. First, the text segmentation module utilizes a pre-trained Bidirectional Encoder Representations from Transformers (BERT) and a bidirectional long short-term memory (LSTM) to divide the input text into segments. An extractive model based on the BERT-based summarization model (BERTSUM) is then constructed to extract the most important sentence from each segment. For training the two-stage summarization model, first, the extracted sentences are used to train the document summarization module in the second stage. Next, the segments are used to train the segment summarization module in the first stage by simultaneously considering the outputs of the segment summarization module and the pre-trained second-stage document summarization module. The parameters of the segment summarization module are updated by considering the loss scores of the document summarization module as well as the segment summarization module. Finally, collaborative training is applied to alternately train the segment summarization module and the document summarization module until convergence. For testing, the outputs of the segment summarization module are concatenated to provide the variable-length abstractive summarization result. For evaluation, the BERT-biLSTM-based text segmentation model is evaluated using ChWiki_181k database and obtains a good effect in capturing the relationship between sentences. Finally, the proposed variable-length abstractive summarization system achieved a maximum of 70.0% accuracy in human subjective evaluation on the LCSTS dataset.
Keyword:
Abstractive summarization
text segmentation
variable-length summarization
BERT
bidirectional LSTM
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
I
IF:
5.1
论文数:
2.6K
被引数:
1.1W
机构
引用论文
Effect of colorspace transformation,the illuminance component,and color modeling on skin detection
OAAI
OAAI分享收藏Extractive Broadcast News Summarization Leveraging Recurrent Neural Network Language Modeling Techniques利用递归神经网络语言建模技术的摘录广播新闻摘要
PREAI
分享收藏A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS).png)
IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏分享收藏Mime: A flexible machine-learning framework to construct and visualize models for clinical characteristics prediction and feature selectionMime: 一个灵活的机器学习框架，用于构建和可视化临床特征预测和特征选择的模型OAAI
OAAI分享收藏
查看更多内容