.png){kind=icon}
.png){kind=icon}
Return
Analyzing how pre-trained language models capture factual knowledge using attribution methods
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2026.115553.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recent research demonstrates that Pre-trained Language Models (PLMs) can correctly complete the cloze-style factual queries like “Dante was born in ____,” showing that PLMs capture factual knowledge through large-scale pre-training data. This phenomenon intrigues researchers to analyze the captured factual knowledge, such as its accuracy, consistency, and biases. This paper investigates from another perspective: how factual knowledge is retained in PLMs as a result of the pre-training process. Specifically, the proposed analysis quantifies the word-level patterns PLMs use to capture factual knowledge from pre-training samples. Two sorts of feature attribution methods, perturbation-based and gradient-based, are used to reveal the dependence on patterns complementarily. Then, the analysis measures how the dependence on different patterns relates to the factual knowledge capture performance, i.e., accuracy and consistency on the cloze-style queries. The analysis results show: (1) PLMs capture the factual knowledge more by the positionally close and highly co-occurred words than the knowledge-dependent words; (2) the dependence on the knowledge-dependent words is more effective than the positionally close and highly co-occurred words. Based on the above observations, the dependence on the inadequate pattern makes the PLMs capture factual knowledge ineffectively. The present analysis reveals the mystery of the factual knowledge capture process, providing empirical observations and discussions, paving the way to improve the pre-training strategies for knowledge-intensive tasks.
Keywords:
factual knowledge
pre-trained language models
feature attribution
cloze-style queries
knowledge capture
Journal
K
IF:
7.6
Papers:
1.2W
Citations:
4.5W