.png){kind=icon}
.png){kind=icon}
Return
Online Learning Algorithms Can Converge Comparably Fast as Batch Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2017.2677970.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Online learning algorithms in a reproducing kernel Hilbert space associated with convex loss functions are studied. We show that in terms of the expected excess generalization error, they can converge comparably fast as corresponding kernel-based batch learning algorithms. Under mild conditions on loss functions and approximation errors, fast learning rates and finite sample upper bounds are established using polynomially decreasing step-size sequences. For some commonly used loss functions for classification, such as the logistic and the p-norm hinge loss functions with p is an element of[1, 2], the learning rates are the same as those for Tikhonov regularization and can be of order O(T-(1/2) log T), which are nearly optimal up to a logarithmic factor. Our novelty lies in a sharp estimate for the expected values of norms of the learning sequence (or an inductive argument to uniformly bound the expected risks of the learning sequence in expectation) and a refined error decomposition for online learning algorithms.
Keywords:
Approximation error
learning theory
online learning
reproducing kernel Hilbert space (RKHS)
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
8.9
Papers:
7.5K
Citations:
7.2W