Optimally regularised kernel Fisher discriminant classification

doi:10.1016/j.neunet.2007.05.005

Optimally regularised kernel Fisher discriminant classification

2007-09-01

19

PRE

AI

DOI：10.1016/j.neunet.2007.05.005

原文链接

原文求助

摘要

En 中文

Mika, Ratsch, Weston, Scholkopf and Muller [Mika, S., Ratsch, G.. Weston, J., Scholkopf, B., & Muller, K.-R. (1999). Fisher discriminant analysis with kernels. In Neural networks for signal processing: Vol. IX (pp. 41-48). New York: IEEE Press] introduce a non-linear formulation of Fisher's linear discriminant, based on the now familiar kernel trick, demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets. In this paper, we extend an existing analytical expression for the leave-one-out cross-validation error [Cawley, G. C., & Talbot, N. L. C. (2003b). Efficient leave-one-out cross-validation of kernel Fisher discriminant classifiers. Pattern Recognition, 36(11), 2585-2592] such that the leave-one-out error can be re-estimated following a change in the value of the regularisation parameter with a computational complexity of only O(l(2)) operations, which is substantially less than the O(l(3)) operations required for the basic training algorithm. This allows the regularisation parameter to be tuned at an essentially negligible computational cost. This is achieved by performing the discriminant analysis in canonical form. The proposed method is therefore a useful component of a model selection strategy for this class of kernel machines that alternates between updates of the kernel and regularisation parameters. Results obtained on real-world and synthetic benchmark datasets indicate that the proposed method is competitive with model selection based on k-fold cross-validation in terms of generalisation, whilst being considerably faster. (C) 2007 Elsevier Ltd. All rights reserved.

Keyword:

model selection

cross-validation

least-squares support vector machine

Optimally regularised kernel Fisher discriminant classification

摘要

AI总结

期刊

机构

引用论文