.png){kind=icon}
.png){kind=icon}
Return
MGCoT : Multi-Grained Contextual Transformer for table-based text generation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2024.123742.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recent advances in Transformer have led to the revolution of table -based text generation. However, most existing Transformer -based architectures ignore the rich contexts among input tokens distributed in multilevel units (e.g., cell, row, or column), leading to sometimes unfaithful text generation that fails to establish accurate association relationships and misses vital information. In this paper, we propose M ulti - G rained Co ntextual T ransformer ( MGCoT ), a novel architecture that fully capitalizes on the multi -grained contexts among input tokens and thus strengthens the capacity of table -based text generation. The key primitive, M ulti - G rained Co ntexts ( MGCo ) module, involves two components: a local context sub -module that adaptively gathers neighboring tokens to form the token -wise local context features, and a global context sub -module that consistently aggregates tokens from a broader range to form the shared global context feature. The former aims at modeling the short-range dependencies that reflect the salience of tokens within similar fine-grained units (e.g., cell and row) attending to the query token, while the latter aims at capturing the long-range dependencies that reflect the significance of each token within similar coarse -grained units (e.g., multiple rows or columns). Based on the fused multi -grained contexts, MGCoT can flexibly and holistically model the content of a table across multi -level structures. On three benchmark datasets, ToTTo, FeTaQA, and Tablesum, MGCoT outperforms strong baselines by a large margin on the quality of the generated texts, demonstrating the effectiveness of multi -grained context modeling. Our source codes are available at https://github.com/Cedric-Mo/MGCoT.
Keywords:
Multi-grained contexts
Transformer
Abstractive table question answering
Table-to-text generation
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W