.png){kind=icon}
返回
Modulo Based Data Placement Algorithm for Energy Consumption Optimization of MapReduce System
.png){kind=icon}
OA
AI
DOI：10.1007/s10723-016-9370-2.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
With the explosion of data production, the efficiency of data management and analysis has been concerned by both industry and academia. Meanwhile, more and more energy is consumed by the IT infrastructure especially the larger scale distributed systems. In this paper, a novel idea for optimizing the Energy Consumption (EC for short) of MapReduce system is proposed. We argue that a fair data placement is helpful to save energy, and then we propose three goals of data placement, and a modulo based Data Placement Algorithm (DPA for short) which achieves these goals. Afterwards, the correctness of the proposed DPA is proved from both theoretical and experimental perspectives. Three different systems which implement MapReduce model with different DPAs are compared in our experiments. Our algorithm is proved to optimize EC effectively, without introducing the additional costs and delaying data loading. With the help of our DPA, the EC for the WordCount can be reduced by 10.9 %, 8.3 % and 17 % respectively, and time consumption is reduced by 7 %, 6.3 % and 7 % respectively.
Keyword:
Big data
Data placement
Energy consumption optimization
MapReduce
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
2.9
论文数:
762
被引数:
1.2K
机构
引用论文
Dynamic energy efficient data placement and cluster reconfiguration algorithm for MapReduce framework面向MapReduce框架的动态节能数据放置和集群重构算法
PREAI
分享收藏CIAlign: A highly customisable command line tool to clean, interpret and visualise multiple sequence alignments.png)
PeerJ
OAAI
PeerJIF0
OAAI分享收藏
查看更多内容