.png){kind=icon}
返回
Hierarchical Concept-Driven Language Model
.png){kind=icon}
PRE
AI
DOI：10.1145/3451167.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
For guiding natural language generation, many semantic-driven methods have been proposed. While clearly improving the performance of the end-to-end training task, these existing semantic-driven methods still have clear limitations: for example, (i) they only utilize shallow semantic signals (e.g., from topic models) with only a single stochastic hidden layer in their data generation process, which suffer easily from noise (especially adapted for short-text etc.) and lack of interpretation; (ii) they ignore the sentence order and document context, as they treat each document as a bag of sentences, and fail to capture the long-distance dependencies and global semantic meaning of a document. To overcome these problems, we propose a novel semantic-driven language modeling framework, which is a method to learn a Hierarchical Language Model and a Recurrent Conceptualization-enhanced Gamma Belief Network, simultaneously. For scalable inference, we develop the auto-encoding Variational Recurrent Inference, allowing efficient end-to-end training and simultaneously capturing global semantics from a text corpus. Especially, this article introduces concept information derived from high-quality lexical knowledge graph Probase, which leverages strong interpretability and anti-nose capability for the proposed model. Moreover, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence concept dependence. Experiments conducted on several NLP tasks validate the superiority of the proposed approach, which could effectively infer meaningful hierarchical concept structure of document and hierarchical multi-scale structures of sequences, even compared with latest state-of-the-art Transformer-based models.
Keyword:
Language modeling
text generation
concept semantic information
interpretation
recurrent conceptualization-enhanced gamma belief network
hierarchical language modeling
representation learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.8
论文数:
1.3K
被引数:
4.4K
机构
引用论文
Chilecomadia valdiviana (Philippi) (Lepidoptera: Cossidae) asociado a Ulmus glabra Hudson forma pendula (Laud.) Rehder ("Olmo pendula") en la VIII Región (Concepción, Chile)Chilecomadia valdiviana (Philippi) (Lepidoptera: Cossidae) 与 Ulmus glabra Hudson forma pendula (Laud.) Rehder ("Olmo pendula") 在第八大区（康塞普西翁，智利）的关联.png)
Bosque
BosqueIF0
PREAI
分享收藏
查看更多内容