.png){kind=icon}
.png){kind=icon}
Return
Tabular reasoning via two-stage knowledge injection
.png){kind=icon}
PRE
AI
DOI：10.1007/s13042-023-02073-4.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Tabular reasoning presents a significant challenge in understanding natural language queries in the context of provided tables, mainly because of the complex logical operations involved. Pre-trained language models have demonstrated their capabilities in various tasks. However, performing pre-training specifically for tabular reasoning is difficult due to the diverse range of reasoning abilities required beyond contextual understanding. In this work, we propose Tabular Reasoning with Two-stage Knowledge Injection (TsKI). TsKI consists of two components: TsKISTAGE1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage1}}$$\end{document} and TsKISTAGE2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage2}}$$\end{document}. The primary objective of TsKISTAGE1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage1}}$$\end{document} is to incorporate symbolic knowledge into pre-trained language models by utilizing synthesized programs. It begins by generating high-quality programs using a specific program synthesis algorithm. Next, TsKISTAGE1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage1}}$$\end{document} conducts pre-training on the automatically generated corpus, enabling the model to learn how to query tables using the generated programs. On the other hand, TsKISTAGE2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage2}}$$\end{document} aims to inject step-wise knowledge into the model. It starts by decomposing natural language queries into multiple sub-queries using heuristic rules and a constituency parser. Then, it employs pre-trained language models themselves to query tables with the obtained sub-queries, obtaining intermediate results that facilitate step-wise tabular reasoning. Experimental results demonstrate the effectiveness of our proposed approach. TsKI achieves significant improvements on two well-known tabular reasoning datasets, namely TabFact and WikiTableQuestions, in both TsKISTAGE1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage1}}$$\end{document} and TsKISTAGE2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_{\textsc {Stage2}}$$\end{document}. Furthermore, in-depth analysis validates the effectiveness of each component of our approach. The code is available at https://github. com/qshi95/TsKI.
Keywords:
Tabular reasoning
Natural language processing
Pre-trained language model
Knowledge injection
Journal
IF:
2.7
Papers:
3.1K
Citations:
5.6K