.png){kind=icon}
.png){kind=icon}
Return
Data coarse graining can improve model performance
.png){kind=icon}
PRE
AI
DOI：10.1103/xn84-qzsw.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Lossy data transformations by definition lose information. Yet, in modern machine learning, methods like data pruning and lossy data augmentation can help improve generalization performance. We study this paradox using a solvable model of high-dimensional, ridge-regularized linear regression under data coarse graining. Conceptually inspired by the renormalization group in statistical physics, we analyze coarse-graining schemes that systematically discard features based on their relevance to the learning task. Our results reveal a nonmonotonic dependence of the prediction risk on the degree of coarse graining. A high-pass scheme-which filters out less relevant, lower-signal features-can help models generalize better. By contrast, a low-pass scheme that integrates out more relevant, higher-signal features is purely detrimental. Crucially, using optimal regularization, we demonstrate that this nonmonotonicity is a distinct effect of data coarse graining and not an artifact of double descent. Our framework offers a clear, analytical explanation for why careful data augmentation works: it strips away less relevant degrees of freedom and isolates more predictive signals. Our results highlight a complex, nonmonotonic risk landscape shaped by the structure of the data and illustrate how ideas from statistical physics provide a principled lens for understanding modern machine learning phenomena.
Keywords:
data coarse graining
model generalization
ridge-regularized linear regression
nonmonotonic risk
statistical physics
Journal
IF:
2.4
Papers:
1.3K
Citations:
10.2W