.png){kind=icon}
返回
Evaluating classifier performance with highly imbalanced Big Data
.png){kind=icon}
OA
AI
DOI：10.1186/s40537-023-00724-5.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Using the wrong metrics to gauge classification of highly imbalanced Big Data may hide important information in experimental results. However, we find that analysis of metrics for performance evaluation and what they can hide or reveal is rarely covered in related works. Therefore, we address that gap by analyzing multiple popular performance metrics on three Big Data classification tasks. To the best of our knowledge, we are the first to utilize three new Medicare insurance claims datasets which became publicly available in 2021. These datasets are all highly imbalanced. Furthermore, the datasets are comprised of completely different data. We evaluate the performance of five ensemble learners in the Machine Learning task of Medicare fraud detection. Random Undersampling (RUS) is applied to induce five class ratios. The classifiers are evaluated with both the Area Under the Receiver Operating Characteristic Curve (AUC), and Area Under the Precision Recall Curve (AUPRC) metrics. We show that AUPRC provides a better insight into classification performance. Our findings reveal that the AUC metric hides the performance impact of RUS. However, classification results in terms of AUPRC show RUS has a detrimental effect. We show that, for highly imbalanced Big Data, the AUC metric fails to capture information about precision scores and false positive counts that the AUPRC metric reveals. Our contribution is to show AUPRC is a more effective metric for evaluating the performance of classifiers when working with highly imbalanced Big Data.
Keyword:
Extremely randomized trees
XGBoost
Class imbalance
Big Data
Undersampling
AUC
AUPRC
期刊
IF:
6.4
论文数:
1.5K
被引数:
1.1W
机构
引用论文
分享收藏分享收藏Severely imbalanced Big Data challenges: investigating data sampling approaches.png)
JOURNAL OF BIG DATA
OAAI
JOURNAL OF BIG DATAIF6.4
OAAI分享收藏Early Diagnosis and Prediction of Sepsis Shock by Combining Static and Dynamic Information Using Convolutional-LSTM基于卷积LSTM的静态和动态信息相结合的脓毒症休克早期诊断和预测
PREAI
分享收藏
查看更多内容