Global Model Selection for Semi-Supervised Support Vector Machine via Solution Paths

doi:10.1109/TNNLS.2024.3354978

2025-02-01

PRE

DOI：10.1109/TNNLS.2024.3354978

原文链接

原文求助

摘要

En 中文

Semi-supervised support vector machine ((SVM)-V-3) is important because it can use plentiful unlabeled data to improve the generalization accuracy of traditional SVMs. In order to achieve good performance, it is necessary for (SVM)-V-3 to take some effective measures to select hyperparameters. However, model selection for semi-supervised models is still a key open problem. Existing methods for semi-supervised models to search for the optimal parameter values are usually computationally demanding, especially those ones with grid search. To address this challenging problem, in this article, we first propose solution paths of (SVM)-V-3 ((SPSVM)-V-3), which can track the solutions of the nonconvex (SVM)-V-3 with respect to the hyperparameters. Specifically, we apply incremental and decremental learning methods to update the solution and let it satisfy the Karush-Kuhn-Tucker (KKT) conditions. Based on the (SPSVM)-V-3 and the piecewise linearity of model function, we can find the model with the minimum cross-validation (CV) error for the entire range of candidate hyperparameters by computing the error path of (SVM)-V-3. Our (SPSVM)-V-3 is the first solution path algorithm for nonconvex optimization problem of semi-supervised learning models. We also provide the finite convergence analysis and computational complexity of (SPSVM)-V-3. Experimental results on a variety of benchmark datasets not only verify that our (SPSVM)-V-3 can globally search the hyperparameters (regularization and ramp loss parameters) but also show a huge reduction of computational time while retaining similar or slightly better generalization performance compared with the grid search approach.

Keyword:

Cross validation (CV)

error path

solution path

Cross validation (CV)

error path

semi-supervised support vector machine ((SVM)-V-3)

solution path