Stability-ranked feature selection for classification in high-dimensional data: combining regularization and machine learning algorithms

doi:10.1007/s40314-026-03681-w

Stability-ranked feature selection for classification in high-dimensional data: combining regularization and machine learning algorithms

2026-04-21

0

PRE

AI

M

Mohammad Kazemi *

A

Amirhossein Khadivi Noghredeh

DOI：10.1007/s40314-026-03681-w

Original

Original request for help

Save

Abstract

En 中文

High-dimensional classification, where the number of features far exceeds the sample size, requires effective and stable feature selection. Penalized logistic regression is a popular choice, but it often produces unstable results that are sensitive to training data splits and tuning parameters. We propose a two-stage method, Frequency-Based Ranking and Incremental Feature Selection, to improve selection stability and classification performance. First, features are ranked by their selection frequencies over N repeated penalized logistic regressions. Second, a chosen classifier is applied using an incremental inclusion of ranked features, with performance evaluated across repeated splits. Simulation studies and real data analyses are conducted to demonstrate the finite-sample performance of the proposed method.

Keywords:

High-dimensional classification

Feature selection

Penalized logistic regression

Support vector machine

Feature ranking

Stability-ranked feature selection for classification in high-dimensional data: combining regularization and machine learning algorithms

Abstract

Journal

Organization