.png){kind=icon}
返回
Multi-Encoder Transformer for Korean Abstractive Text Summarization
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2023.3277754.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this paper, we propose a Korean abstractive text summarization approach that uses a multi -encoder transformer. Recently, in many natural language processing (NLP) tasks, the use of the pre-trained language models (PLMs) for transfer learning has achieved remarkable performance. In particular, transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT) are used for pre-training and applied to downstream tasks, showing state-of-the-art performance including abstractive text summarization. However, existing text summarization models usually use one pre-trained model per model architecture, meaning that it becomes necessary to choose one PLM at a time. For PLMs applicable to Korean abstractive text summarization, there are publicly available BERT-based pre-trained Korean models that offer different advantages such as Multilingual BERT, KoBERT, HanBERT, and KorBERT. We assume that if these PLMs could be leveraged simultaneously, better performance would be obtained. We propose a model that uses multiple encoders which are capable of leveraging multiple pre-trained models to create an abstractive summary. We evaluate our method using three benchmark Korean abstractive summarization datasets, each named Law (AI-Hub), News (AI-Hub), and News (NIKL) datasets. Experimental results show that the proposed multi-encoder model variations outperform single -encoder models. We find the empirically best summarization model by determining the optimal input combination when leveraging multiple PLMs with the multi-encoder method.
Keyword:
Natural language processing
abstractive text summarization
bidirectional encoder repre-sentations from transformer
neural networks
natural language generation
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Responses of isomeric floridosides under stress in two heteromorphic generations of Neoporphyra haitanensis两种异形世代石莼科藻类Neoporphyra haitanensis中同分异构体岩藻糖苷在胁迫下的响应
PREAI
分享收藏The flowering plants of Africa; an analytical key to the genera of African phanerogams非洲开花植物；非洲显花植物科属的分析检索表.png)
OAAI
IF0
OAAI分享收藏
查看更多内容