Multi-Document Summarization Using Selective Attention Span and Reinforcement Learning

Yash Kumar Atri

doi:10.1109/TASLP.2023.3316459

Return

Multi-Document Summarization Using Selective Attention Span and Reinforcement Learning

2023-01-01

PRE

Yash Kumar Atri *

Vikram Goyal

Tanmoy Chakraborty

DOI：10.1109/TASLP.2023.3316459

Original

Original request for help

Save

Abstract

En 中文

The Abstractive text summarization systems using recently improved RNN-based sequence-to-sequence architecture have shown great promise for single-document summarization. However, such neural models fail to perpetuate the performance in the multi-document summarization setting owing to the long-range dependencies within the documents, overlapping/contradicting facts and extrinsic model hallucinations. These shortcomings augment the model to generate inconsistent, repetitive and non-factual summaries. In this work, we introduce REISA, a sequence-to-sequence model with a novel reinforced selective attention span that attends over the input and recalibrates the local attention weights to focus on important segments while generating output at each time step. REISA utilizes a reinforcement learning-based policy gradient algorithm to reward the model and formulate attention distributions over the encoder input. We further benchmark REISA on two widely-used multi-document summarization corpora - Multinews and CQASumm, and observe an improvement of +2.91 and +6.64 ROUGE-L scores, respectively. The qualitative analyses on semantic similarity by BERTScore, faithfulness by question-answer evaluation and human evaluation show significant improvement over the baseline-generated summaries.

Keywords:

Abstractive Text Summarization

Multi-Document Summarization

seq2seq

Deep Reinforcement Learning