.png){kind=icon}
返回
Multi-aspect permutation tests for model selection
.png){kind=icon}
OA
AI
DOI：10.1111/exsy.13492.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In typical machine learning frameworks, model selection is of fundamental importance: commonly, multiple models have to be trained and compared in order to identify the one with the best predictive performances. The aim of this study is to provide a new tool to improve the model selection process, allowing the user to identify the algorithm which significantly outperforms the other candidates. It proposes a robust model selection procedure based on a multi-aspect permutation test which makes it possible to detect differences in both location and variability between two paired samples of prediction errors. A new extension of the nonparametric combination (NPC) methodology is therefore introduced and is integrated with an appropriate ranking procedure in order to deal with the comparison of C >= 2$$ C\ge 2 $$ candidate models. A simulation study is conducted to evaluate the performances of this testing procedure in 2-sample and C$$ C $$-sample problems, by generating data from various well-known distributions and simulating several possible null and alternative scenarios. The adoption of the proposed technique in machine learning model selection problems is then discussed by means of multiple real data applications. These applications confirm what emerges from the simulation study: the introduced NPC-based approach performs well under several different scenarios and represents a valuable tool for robust machine learning model selection.
Keyword:
machine learning
model selection
multi-aspect
permutation test
期刊
IF:
2.3
论文数:
2.6K
被引数:
3.8K
机构
引用论文
.png){kind=icon}
Cyber Security Threats to Bitcoin Exchanges: Adversary Exploitation and Laundering Techniques比特币交易所面临的网络安全威胁：对手方利用与洗钱技术
PREAI
分享收藏One size does not fit all: security hardening of MIPS embedded systems via static binary debloating for shared libraries一种尺寸并不适合所有情况：通过静态二进制精简为共享库的MIPS嵌入式系统安全加固OAAI
OAAI分享收藏
查看更多内容