Effective Automatic Method Selection for Nonlinear Regression Modeling

Jan Kalina

doi:10.1142/S0129065721500209

Return

Effective Automatic Method Selection for Nonlinear Regression Modeling

2021-03-29

PRE

Jan Kalina

Aleš Neoral

Petra Vidnerová *

DOI：10.1142/S0129065721500209

Original

Save

Abstract

En 中文

Metalearning, an important part of artificial intelligence, represents a promising approach for the task of automatic selection of appropriate methods or algorithms. This paper is interested in recommending a suitable estimator for nonlinear regression modeling, particularly in recommending either the standard nonlinear least squares estimator or one of such available alternative estimators, which is highly robust with respect to the presence of outliers in the data. The authors hold the opinion that theoretical considerations will never be able to formulate such recommendations for the nonlinear regression context. Instead, metalearning is explored here as an original approach suitable for this task. In this paper, four different approaches for automatic method selection for nonlinear regression are proposed and computations over a training database of 643 real publicly available datasets are performed. Particularly, while the metalearning results may be harmed by the imbalanced number of groups, an effective approach yields much improved results, performing a novel combination of supervised feature selection by random forest and oversampling by synthetic minority oversampling technique (SMOTE). As a by-product, the computations bring arguments in favor of the very recent nonlinear least weighted squares estimator, which turns out to outperform other (and much more renowned) estimators in a quite large percentage of datasets.

Keywords:

Metalearning

nonlinear regression

robust statistical estimation

feature selection

AutoML