Performance of some variable selection methods when multicollinearity is present

Il-Gyo Chong

doi:10.1016/j.chemolab.2004.12.011

2005-07-01

1.7K

PRE

Il-Gyo Chong

Chi‐Hyuck Jun

DOI：10.1016/j.chemolab.2004.12.011

原文链接

原文求助

摘要

En 中文

Variable selection is one of the important practical issues for many scientific engineers. Although the PLS (partial least squares) regression combined with the VIP (variable importance in the projection) scores is often used when the multicollinearity, is present among variables, there are few guidelines about its uses as well as its performance. The purpose of this paper is to explore the nature of the VIP method and to compare with other methods through computer simulation experiments. We design 108 experiments where observations are generated from true models considering four factors-the proportion of the number of relevant predictors, the magnitude of correlations between predictors, the structure of regression coefficients, and the magnitude of signal to noise. Confusion matrix is adopted to evaluate the performance of PLS, the Lasso, and stepwise method. We also discuss the proper cutoff value of the VIP method to increase its performance. Some practical hints for the use of the VIP method are given as simulation results. (c) 2005 Elsevier B.V. All rights reserved.

Keyword:

variable selection

VIP (Variable importance in the projection) scores

partial least squares regression

the lasso

stepwise regression

multicollinearity