Model building using forward selection and machine learning

doi:10.1080/02664763.2026.2625120

Return

Model building using forward selection and machine learning

2026-02-01

PRE

Biris, Panagiotis

Economou, Polychronis *

DOI：10.1080/02664763.2026.2625120

Original

Original request for help

Save

Abstract

En 中文

A common challenge in large-scale data sets with numerous predictors is multicollinearity, which can degrade the performance and interpretability of predictive models. Although dimensionality reduction techniques like Principal Component Analysis (PCA) can mitigate multicollinearity, they transform original variables into new uncorrelated variables, often reducing the model interpretability. For this reason, feature selection techniques are generally preferred in domains where understanding of the original predictors is essential. This paper presents a novel feature selection method that combines machine learning techniques with forward feature selection to effectively address multicollinearity while preserving the accuracy of prediction. The proposed method is evaluated on three diverse data sets from the maritime, healthcare, and environmental domains. Regression models were used to assess predictive performance based on adjusted $ R<^>2 $ R2, Root Mean Square Error and Mean Absolute Error. The results demonstrate that the proposed method achieves high performance comparable to traditional feature selection techniques while preventing multicollinearity. This balance between predictive power and feature stability makes it especially suitable for high-dimensional data sets, where redundant or highly correlated variables are prevalent. The approach supports more robust and interpretable models, highlighting its potential for widespread application in complex, data-intensive domains.

Keywords:

Feature selection

multicollinearity

machine learning

regression

real-world data