.png){kind=icon}
.png){kind=icon}
Return
Discourse-Aware Language Representation
.png){kind=icon}
PRE
AI
DOI：10.1109/TPAMI.2025.3621229.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recent Transformer-based language representation techniques have commonly adopted a straightforward approach to modeling textual context as a linear sequence of successive tokens. However, this sequential modeling strategy falls short in actively exploring intermediate structures present in natural languages and does not account for the rich interactive relationships between sentences. To overcome these limitations, we propose a discourse-aware framework that bridges the gap between sequential contextualization and the interactive nature of conversational reading comprehension. Concretely, we first divide the context into elementary discourse units (EDUs), ensuring that each unit contains precisely one condition. Then, we systematically explore three instantiations for modeling discourse features: sequential EDU encoding, discourse-aware masking, and discourse graph network. These techniques allow us to capture the nuanced interactions within the discourse. To assess the efficacy of our methodologies, we perform experiments on three conversational reading comprehension tasks: multi-turn response selection, conversational question answering, and conversational machine reading. Experimental results demonstrate the superiority of our proposed approach. Moreover, analysis reveals that the discourse-aware approach enables the model to effectively capture intricate relationships within the context and fosters reasoning interpretability. Additionally, our method exhibits efficacy across various backbone PLMs and diverse domains.
Keywords:
Natural language processing
language representation
large language models
conversational dialogue modeling
natural language generation
response selection
reading comprehension
conversational question answering
deep neural networks
graph convolutional networks
chatGPT
Journal
IF:
18.6
Papers:
831
Citations:
9.8W