.png){kind=icon}
.png){kind=icon}
Return
Linguistic structure from a bottleneck on sequential information processing
.png){kind=icon}
OA
AI
DOI：10.1038/s41562-025-02336-w.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Human language has a distinct systematic structure, where utterances break into individually meaningful words that are combined to form phrases. Here we show that natural-language-like systematicity arises in codes that are constrained by a statistical measure of complexity called predictive information, also known as excess entropy. Predictive information is the mutual information between the past and future of a stochastic process. In simulations, we find that codes that minimize predictive information break messages into groups of approximately independent features that are expressed systematically and locally, corresponding to words and phrases. Next, drawing on cross-linguistic text corpora, we find that actual human languages are structured in a way that yields low predictive information compared with baselines at the levels of phonology, morphology, syntax and lexical semantics. Our results establish a link between the statistical and algebraic structure of language and reinforce the idea that these structures are shaped by communication under general cognitive constraints. Futrell and Hahn argue that languages can be understood as codes that minimize predictive information; that is, they express approximately independent features systematically and locally.
Keywords:
predictive information
linguistic structure
systematicity
communication
cognitive constraints
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
15.9
Papers:
460
Citations:
1.7W