Asynchronous Parallel Large-Scale Gaussian Process Regression

Zhiyuan Dang

doi:10.1109/TNNLS.2022.3200602

Asynchronous Parallel Large-Scale Gaussian Process Regression

2024-06-01

1

PRE

AI

DOI：10.1109/TNNLS.2022.3200602

Original

Original request for help

Save

Abstract

En 中文

Gaussian process regression (GPR) is an important nonparametric learning method in machine learning research with many real-world applications. It is well known that training large-scale GPR is a challenging task due to the required heavy computational cost and large volume memory. To address this challenging problem, in this article, we propose an asynchronous doubly stochastic gradient algorithm to handle the large-scale training of GPR. We formulate the GPR to a convex optimization problem, i.e., kernel ridge regression. After that, in order to efficiently solve this convex kernel problem, we first use the random feature mapping method to approximate the kernel model and then utilize two unbiased stochastic approximations, i.e., stochastic variance reduced gradient and stochastic coordinate descent, to update the solution asynchronously and in parallel. In this way, our algorithm scales well in both sample size and dimensionality, and speeds up the training computation. More importantly, we prove that our algorithm has a global linear convergence rate. Our experimental results on eight large-scale benchmark datasets with both regression and classification tasks show that the proposed algorithm outperforms the existing state-of-the-art GPR methods.

Keywords:

Kernel

Approximation algorithms

Training

Optimization

Gaussian processes

Convergence

Computational modeling

Asynchronous parallel learning

big data

Gaussian process regression (GPR)

random features

Asynchronous Parallel Large-Scale Gaussian Process Regression

Abstract

Journal

Organization