Gaussian Universality for Diffusion Models

doi:10.1109/LSP.2025.3637667

Gaussian Universality for Diffusion Models

2025-12-19

0

PRE

AI

DOI：10.1109/LSP.2025.3637667

Original

Original request for help

Save

Abstract

En 中文

We investigate Gaussian Universality for data distributions generated via diffusion models. By Gaussian Universality we mean that the test error of a generalized linear model $f(\mathbf {W})$ trained for a classification task on the diffusion data matches the test error of $f(\mathbf {W})$ trained on the Gaussian Mixture with matching means and covariances per class. In other words, the test error depends only on the first and second order statistics of the diffusion-generated data in the linear setting. As a corollary, the analysis of the test error for linear classifiers can be reduced to Gaussian data from diffusion-generated data. Analyzing the performance of models trained on synthetic data is a pertinent problem due to the surge of methods such as [Sehwag, et al. (2024)]. Moreover, we show that, for any 1- Lipschitz scalar function $\phi$, $\phi (\mathbf {x})$ is close to $\mathbb {E}\phi (\mathbf {x})$ with high probability for $\mathbf {x}$ sampled from the conditional diffusion model corresponding to each class. Finally, we note that current approaches for proving universality do not apply to diffusion-generated data as the covariance matrices of the data tend to have vanishing minimum singular values, contrary to the assumption made in the literature. This leaves extending previous mathematical universality results as an intriguing open question.

Keywords:

Gaussian universality

diffusion models

multiclass classification

generalized linear models

Gaussian Universality for Diffusion Models

Abstract

Journal

Organization

Cited Papers