.png){kind=icon}
.png){kind=icon}
Return
Model validation using mutated training labels: An exploratory study
.png){kind=icon}
OA
AI
DOI：10.1016/j.neucom.2023.02.042.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We introduce an exploratory study on Mutation Validation (MV), a model validation method using mutated training labels for supervised learning. MV mutates training data labels, retrains the model against the mutated data, and then uses the metamorphic relation that captures the consequent training performance changes to assess model fit. It does not use a validation set or test set. The intuition under-pinning MV is that overfitting models tend to fit noise in the training data.MV does not aim to replace out-of-sample validation. Instead, we provide the first exploratory study on the possibility of using MV as a complement of out-of-sample validation. We explore 8 different learning algorithms, 18 datasets, and 5 types of hyperparameter tuning tasks. Our results demonstrate that MV complements well cross-validation and test accuracy in model selection and hyperparameter tuning tasks. MV deserves more attention from developers when simplicity, sustainaiblity, security (e.g., defend-ing training data attack), and interpretability of the built models are required.(c) 2023 Published by Elsevier B.V.
Keywords:
Model validation
Model complexity
Model overfitting
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W