Active model selection: A variance minimization approach

Satoshi Hara

doi:10.1007/s10994-024-06603-1

Active model selection: A variance minimization approach

2024-11-21

0

OA

AI

DOI：10.1007/s10994-024-06603-1

Original

Original request for help

Save

Abstract

En 中文

The cost of labeling is a significant challenge in practical machine learning. This issue arises not only during the learning phase but also at the model evaluation phase, as there is a need for a substantial amount of labeled test data in addition to the training data. In this study, we address the challenge of active model selection with the goal of minimizing labeling costs for choosing the best-performing model from a set of model candidates. Based on an appropriate test loss estimator, we propose an adaptive labeling strategy that can estimate the difference of test losses with small variance, thereby enabling the estimation of the best model using fewer labeling cost. Experimental results on real-world datasets confirm that our method efficiently selects the best model.

Keywords:

Supervised learning

Active learning

Model selection

Active testing

Active model selection: A variance minimization approach

Abstract

Journal

Organization