Classifier learning with a new locality regularization method

Hui Xue

doi:10.1016/j.patcog.2007.09.016

Classifier learning with a new locality regularization method

2008-05-01

8

PRE

AI

H

Hui Xue

Songcan Chen (Songcan Chen) *

X

Xiaoqin Zeng

DOI：10.1016/j.patcog.2007.09.016

Original

Original request for help

Save

Abstract

En 中文

It is well known that the generalization capability is one of the most important criterions to develop and evaluate a classifier for a given pattern classification problem. The localized generalization error model (R-SM) recently proposed by Ng et al. [Localized generalization error and its application to RBFNN training, in: Proceedings of the International Conference on Machine Learning and Cybernetics, China, 2005; Image classification with the use of radial basis function neural networks and the minimization of the localized generalization error, Pattern Recognition 40(l) (2007) 4-18] provides a more intuitive look at the generalization error. Although R-SM gives a brand-new method to promote the generalization performance, it is in nature equivalent to another type of regularization. In this paper, we first prove the essential relationship between R-SM and regularization, and demonstrate that the stochastic sensitivity measure in R-SM exactly corresponds to a regularizing term. Then, we develop a new generalization error bound from the regularization viewpoint, which is inspired by the proved relationship between R-SM and regularization. Moreover, we derive a new regularization method, called as locality regularization (LR), from the bound. Different from the existing regularization methods which artificially and externally append the regularizing term in order to smooth the solution, LR is naturally and internally deduced from the defined expected risk functional and calculated by employing locality information. Through combining with spectral graph theory, LR introduces the local structure information of the samples into the regularizing term and further improves the generalization capability. In contrast with R-SM, which is relatively sensitive to the different sampling of the samples, LR uses the discrete k-neighborhood rather than the common continuous Q-neighborhood in R-SM to differentiate the relative position of different training samples automatically and avoid the complex computation of Q for various classifiers. Furthermore, LR uses the regularization parameter to control the trade-off between the training accuracy and the classifier stability. Experimental results on artificial and real world problems show that LR yields better generalization capability than both R-SM and some traditional regularization methods. (c) 2007 Elsevier Ltd. All rights reserved.

Keywords:

localized generalization error model

stochastic sensitivity measure

locality regularization (LR)

classifier learning

pattern classification

Classifier learning with a new locality regularization method

Abstract

AI Summary

Journal

Organization

Cited Papers