.png){kind=icon}
Return
Scatter-Gather-Merge: An efficient star-join query processing algorithm for data-parallel frameworks
.png){kind=icon}
PRE
AI
DOI：10.1007/s10586-010-0144-5.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
A data-parallel framework is very attractive for large-scale data processing since it enables such an application to easily process a huge amount of data on commodity machines. MapReduce, a popular data-parallel framework, is used in various fields such as web search, data mining and data warehouses; it is proven to be very practical for such a data-parallel application. A star-join query is a popular query in data warehouses that are a current target domain of data-parallel frameworks. This article proposes a new algorithm that efficiently processes star-join queries in data-parallel frameworks such as MapReduce and Dryad. Our star-join algorithm for general data-parallel frameworks is called Scatter-Gather-Merge, and it processes star-join queries in a constant number of computation steps, although the number of participating dimension tables increases. By adopting bloom filters, Scatter-Gather-Merge reduces a non-trivial amount of IO. We also show that Scatter-Gather-Merge can be easily applied to MapReduce. Our experimental results in both cluster and cloud environments show that Scatter-Gather-Merge outperforms existing approaches.
Keywords:
Data parallel framework
MapReduce
Hadoop
Star-join query
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
C
IF:
4.1
Papers:
5.1K
Citations:
7.5K
Organization
Cited Papers
ShareSaveShareSave
Concentrations of myo-inositol in the luminal fluid of the mammalian testis and epididymis.png)
Reproduction
OAAI
ReproductionIF0
OAAIShareSaveShareSaveCharacteristics and modelling of BDS satellite inter-frequency clock bias for triple-frequency PPP
PREAI
ShareSave no more
no more