An information theoretic sparse kernel algorithm for online learning

doi:10.1016/j.eswa.2014.01.010

Return

An information theoretic sparse kernel algorithm for online learning

2014-07-01

PRE

Haijin Fan *

Qing Song

Xu Zhao

DOI：10.1016/j.eswa.2014.01.010

Original

Original request for help

Save

Abstract

En 中文

Kernel-based algorithms have been proven successful in many nonlinear modeling applications. However, the computational complexity of classical kernel-based methods grows superlinearly with the increasing number of training data, which is too expensive for online applications. In order to solve this problem, the paper presents an information theoretic method to train a sparse version of kernel learning algorithm. A concept named instantaneous mutual information is investigated to measure the system reliability of the estimated output. This measure is used as a criterion to determine the novelty of the training sample and informative ones are selected to form a compact dictionary to represent the whole data. Furthermore, we propose a robust learning scheme for the training of the kernel learning algorithm with an adaptive learning rate. This ensures the convergence of the learning algorithm and makes it converge to the steady state faster. We illustrate the performance of our proposed algorithm and compare it with some recent kernel algorithms by several experiments. (C) 2014 Elsevier Ltd. All rights reserved.

Keywords:

Kernel methods

Information theoretic

Sparsification

Online learning

Mutual information

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.