.png){kind=icon}
.png){kind=icon}
Return
SCALABLE INFERENCE FOR NONPARAMETRIC STOCHASTIC APPROXIMATION IN REPRODUCING KERNEL HILBERT SPACES
.png){kind=icon}
PRE
AI
L
DOI：10.1214/25-AOS2587.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Stochastic approximation (SA) is a powerful and scalable computational method for iteratively estimating the solution of optimization problems in the presence of randomness, particularly well suited for large-scale and streaming data settings. In this work we propose a theoretical framework for stochastic approximation (SA) applied to nonparametric least squares in reproducing kernel Hilbert spaces (RKHS), enabling online statistical inference in non-parametric regression models. We achieve this by constructing asymptotically valid pointwise (and simultaneous) confidence intervals (bands) for local (and global) inference of the nonlinear regression function, via employing an online multiplier bootstrap approach to a functional stochastic gradient descent (SGD) algorithm in the RKHS. Our main theoretical contributions consist of a unified framework for characterizing the nonasymptotic behavior of the functional SGD estimator and demonstrating the consistency of the multiplier bootstrap method. The proof techniques involve the development of a higher-order expansion of the functional SGD estimator under the supremum norm metric and the Gaussian approximation of suprema of weighted and non-identically distributed empirical processes. Our theory specifically reveals an interesting relationship between the tuning of step sizes in SGD for estimation and the accuracy of uncertainty quantification.
Keywords:
Bootstrap
functional stochastic gradient descent
nonparametric regression
online inference
RKHS
Journal
IF:
3.7
Papers:
2.8K
Citations:
2.9W
Organization
Cited Papers