.png){kind=icon}
返回
Balancing reducer workload for skewed data using sampling-based partitioning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.compeleceng.2013.07.001.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
MapReduce has emerged as a popular tool for distributed processing of massive data. However, it is not efficient when handling skewed data and it often leads to reducer load imbalance. In this paper, we address the problem of how to efficiently partition intermediate keys to balance the workload of all reducers when processing skewed data. We present a sampling scheme to compute the approximate distribution of key frequency, estimate the overall distribution and then make a partition scheme in advance. Then, we apply it to map phase of the executing MapReduce job. This work not only provides a load-balanced partition strategy, but also keeps a high performance of synchronous mode of MapReduce. We also propose two partition methods based on sampling results: cluster combination and cluster split combination. The experimental results show that our methods achieve a better time and load balancing results. (C) 2013 Elsevier Ltd. All rights reserved.
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
C
IF:
4.9
论文数:
6.7K
被引数:
1.3W
机构
引用论文
.png){kind=icon}
Semen Donors as the Source of Sexually Transmitted Diseases in Artificially Inseminated Women: The Saga Unfolds精液捐赠者是人工授精妇女性传播疾病的来源: 传奇故事展开
PREAI
分享收藏分享收藏GERMINAL CENTER AND NON- GERMINAL CENTER B CELL RESPONSE TO FACTOR VIII IN HEMOPHILIA A PATIENTS血友病A患者中，生发中心与非生发中心B细胞对因子VIII的应答
PREAI
分享收藏PDK2 induces cisplatin-resistance in lung adenocarcinoma via transcriptional regulation of CNNM3
PREAI
分享收藏
查看更多内容