.png){kind=icon}
.png){kind=icon}
Return
Benchmarking big data architectures for social networks data processing using public cloud platforms
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2018.05.068.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
When considering popular On-line Social Networks (OSN) containing heterogeneous multimedia data sources, the complexity of the underlying processing systems becomes challenging, and requires to implement application-specific but still comprehensive benchmarking. The variety of big data architectures (and of their possible realization) for both batch and streaming processing in a huge number of application domains, makes the benchmarking of these systems critical for both academic and industrial communities. In this work, we evaluate the performance of two state-of-art big data architectures, namely Lambda and Kappa, considering OSN data analysis as reference task. In more details, we have implemented and deployed an influence analysis algorithm on the Microsoft Azure public cloud platform to investigate the impact of a number of factors on the performance obtained by cloud users. These factors comprise the type of the implemented architecture, the volume of the data to analyze, the size of the cluster of nodes realizing the architectures and their characteristics, the deployment costs, as well as the quality of the output when the analysis is subjected to strict temporal deadlines. Experimental campaigns have been carried out on the Yahoo Flickr Creative Commons 100 Million (YFCC100M). Reported results and discussions show that Lambda outperforms Kappa architecture for the class of problems investigated. Providing a variety of analyses - e.g., also investigating the impact of dataset size, scaling, cost - this paper provides useful insights on the performance of these state-of-art big data architectures that are helpful to both experts and newcomers interested in deploying big data architectures leveraging cloud platforms. (C) 2018 Elsevier B.V. All rights reserved.
Keywords:
On-line social networks
lambda and Kappa architectures
Influence analysis
Social networking benchmarking
Public cloud benchmarking
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
F
IF:
6.1
Papers:
6.8K
Citations:
2.3W