.png){kind=icon}
.png){kind=icon}
Return
Multikernel Passive Stochastic Gradient Algorithms and Transfer Learning
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2021.3079280.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This article develops a novel passive stochastic gradient algorithm. In passive stochastic approximation, the stochastic gradient algorithm does not have control over the location where noisy gradients of the cost function are evaluated. Classical passive stochastic gradient algorithms use a kernel that approximates a Dirac delta to weigh the gradients based on how far they are evaluated from the desired point. In this article, we construct a multikernel passive stochastic gradient algorithm. The algorithm performs substantially better in high dimensional problems and incorporates variance reduction. We analyze the weak convergence of the multikernel algorithm and its rate of convergence. In numerical examples, we study the multikernel version of the passive least mean squares algorithm for transfer learning to compare the performance with the classical passive version.
Keywords:
Convergence
Approximation algorithms
Kernel
Noise measurement
Transfer learning
Monte Carlo methods
Ordinary differential equations
Bernstein von-Mises theorem
passive least mean squares (LMS)
stochastic gradient algorithm
stochastic sampling
transfer learning
variance reduction
weak convergence
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
7
Papers:
1.3W
Citations:
6.7W