A Dynamic Spark-based Classification Framework for Imbalanced Big Data

Nahla B. Abdel-Hamid

doi:10.1007/s10723-018-9465-z

Return

A Dynamic Spark-based Classification Framework for Imbalanced Big Data

2018-09-01

PRE

Nahla B. Abdel-Hamid

Sally M. Elghamrawy *

El Desouky, Ali

Arafat, Hesham

DOI：10.1007/s10723-018-9465-z

Original

Original request for help

Save

Abstract

En 中文

Classification of imbalanced big data has assembled an extensive consideration by many researchers during the last decade. Standard classification methods poorly diagnosis the minority class samples. Several approaches have been introduced for solving the problem of class imbalance in big data to enhance the generalization in classification. However, most of these approaches neglect the effect of border samples on classification performance; the high impact border samples might expose to misclassification. In this paper, a Spark Based Mining Framework (SBMF) is proposed to address the imbalanced data problem. Two main modules are designed for this purpose. The first is the Border Handling Module (BHM) which under samples the low impact majority border instances and oversamples the minority class instances. The second module is the Selective Border Instances sampling (SBI) Module, which enhances the output of the BHM module. The performance of the SBMF framework is evaluated and compared with other recent systems. A number of experiments were performed using moderate and big datasets with different imbalanced ratio. The results obtained from SBMF framework, when compared to the recent works, show better performance for the different datasets and classifiers.

Keywords:

Big data

Classification

Spark

Sampling methods

Borders

Imbalanced datasets

SMOTE

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.