.png){kind=icon}
返回
Multi-class imbalanced big data classification on Spark
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2020.106598.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Despite more than two decades of progress, learning from imbalanced data is still considered as one of the contemporary challenges in machine learning. This has been further complicated by the advent of the big data era, where popular algorithms dedicated to alleviating the class skew impact are no longer feasible due to the volume of datasets. Additionally, most of existing algorithms focus on binary imbalanced problems, where majority and minority classes are well-defined. Multi-class imbalanced data poses further challenges as the relationship between classes is much more complex and simple decomposition into a number of binary problems leads to a significant loss of information. In this paper, we propose the first compound framework for dealing with multi-class big data problems, addressing at the same time the existence of multiple classes and high volumes of data. We propose to analyze the instance-level difficulties in each class, leading to understanding what causes learning difficulties. We embed this information in popular resampling algorithms which allows for informative balancing of multiple classes. We propose an efficient implementation of the discussed algorithm on Apache Spark, including a novel version of SMOTE that overcomes spatial limitations in distributed environments of its predecessor. Extensive experimental study shows that using instance-level information significantly improves learning from multi-class imbalanced big data. Our framework can be downloaded from https://github.com/fsleeman/minority-type-imbalanced. (C) 2020 Elsevier B.V. All rights reserved.
Keyword:
Machine learning
Big data
Imbalanced data classification
Multi-class imbalance
Spark
MapReduce
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
K
IF:
7.6
论文数:
1.2W
被引数:
4.5W
机构
引用论文
Radial-Based oversampling for noisy imbalanced data classification基于径向过采样的噪声不平衡数据分类.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏A simple plug-in bagging ensemble based on threshold-moving for classifying binary and multiclass imbalanced dataNEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏A study on combining dynamic selection and data preprocessing for imbalance learningNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容