Big data processing using hybrid Gaussian mixture model with salp swarm algorithm

doi:10.1186/s40537-024-01015-3

Big data processing using hybrid Gaussian mixture model with salp swarm algorithm

2024-11-21

0

OA

AI

Mesfin Esayas Lelisho *

DOI：10.1186/s40537-024-01015-3

原文链接

原文求助

摘要

En 中文

The traditional methods used in big data, like cluster creation and query-based data extraction, fail to yield accurate results on massive networks. To address such issues, the proposed approach involves using the Hadoop Distributed File System (HDFS) for data processing, the map-reduce programming paradigm for data processing, and query optimization techniques to quickly and effectively extract accurate outcomes from a variety of options with a high processing capacity. The methodology proposed in this work makes use of Gaussian Mixture Model (GMM) for data clustering and the Salp Swarm Algorithm (SSA) for optimization. The security of preprocessed data stored on networked clusters with interconnections has been ensured by SHA algorithms. Finally, incorporating into consideration the important parameters, evaluation findings for the experimental performance of the model in the indicated methodology are produced. For this work, the estimated range of input file sizes is 60-100 MB. The processing of 100 MB of input files yielded an accuracy of 96% and results for specificity and sensitivity of 90% and 93%, respectively. The outcomes have been compared with well-known methods like fuzzy C-means and K-means approaches, and the results show that the proposed method effectively distributes accurate data processing to cluster nodes with low latency. Moreover, it uses the least amount of memory resources possible when operating on functional CPUs. As a result, the proposed approach outperforms existing techniques.

Keyword:

Hadoop distributed file system (HDFS)

Map-reduce

Gaussian mixture model (GMM)

Salp swarm algorithm (SSA)

Secure hash algorithms (SHA)

Big data processing using hybrid Gaussian mixture model with salp swarm algorithm

摘要

期刊

机构

引用论文