Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis

Wendelin Böhmer

doi:10.1007/s10994-012-5300-0

Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis

2012-06-13

17

OA

AI

DOI：10.1007/s10994-012-5300-0

原文链接

原文求助

摘要

En 中文

Without non-linear basis functions many problems can not be solved by linear algorithms. This article proposes a method to automatically construct such basis functions with slow feature analysis (SFA). Non-linear optimization of this unsupervised learning method generates an orthogonal basis on the unknown latent space for a given time series. In contrast to methods like PCA, SFA is thus well suited for techniques that make direct use of the latent space. Real-world time series can be complex, and current SFA algorithms are either not powerful enough or tend to over-fit. We make use of the kernel trick in combination with sparsification to develop a kernelized SFA algorithm which provides a powerful function class for large data sets. Sparsity is achieved by a novel matching pursuit approach that can be applied to other tasks as well. For small data sets, however, the kernel SFA approach leads to over-fitting and numerical instabilities. To enforce a stable solution, we introduce regularization to the SFA objective. We hypothesize that our algorithm generates a feature space that resembles a Fourier basis in the unknown space of latent variables underlying a given real-world time series. We evaluate this hypothesis at the example of a vowel classification task in comparison to sparse kernel PCA. Our results show excellent classification accuracy and demonstrate the superiority of kernel SFA over kernel PCA in encoding latent variables.

Keyword:

Time series

Latent variables

Unsupervised learning

Slow feature analysis

Sparse kernel methods

Linear classification

Generating feature spaces for linear algorithms with regularized sparse kernel slow feature analysis

摘要

期刊

机构