.png){kind=icon}
返回
Scalable parallel computing on clouds using Twister4Azure iterative MapReduce
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2012.05.027.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recent advances in data-intensive computing for science discovery are fueling a dramatic growth in the use of data-intensive iterative computations. The utility computing model introduced by cloud computing, combined with the rich set of cloud infrastructure and storage services, offers a very attractive environment in which scientists can perform data analytics. The challenges to large-scale distributed computations on cloud environments demand innovative computational frameworks that are specifically tailored for cloud characteristics to easily and effectively harness the power of clouds. Twister4Azure is a distributed decentralized iterative MapReduce runtime for Windows Azure Cloud. Twister4Azure extends the familiar, easy-to-use MapReduce programming model with iterative extensions, enabling a fault-tolerance execution of a wide array of data mining and data analysis applications on the Azure cloud. Twister4Azure utilizes the scalable, distributed and highly available Azure cloud services as the underlying building blocks, and employs a decentralized control architecture that avoids single point failures. Twister4Azure optimizes the iterative computations using a multi-level caching of data, a cache aware decentralized task scheduling, hybrid tree-based data broadcasting and hybrid intermediate data communication. This paper presents the Twister4Azure iterative MapReduce runtime and a study of four real world data-intensive scientific applications implemented using Twister4Azure two iterative applications, Multi-Dimensional Scaling and KMeans Clustering; and two pleasingly parallel applications, BLAST+ sequence searching and SmithWaterman sequence alignment. Performance measurements show comparable or a factor of 2 to 4 better results than the traditional MapReduce runtimes deployed on up to 256 instances and for jobs with tens of thousands of tasks. We also study and present solutions to several factors that affect the performance of iterative MapReduce applications on Windows Azure Cloud, (c) 2012 Elsevier B.V. All rights reserved,
Keyword:
Iterative MapReduce
Cloud computing
HPC
Scientific applications
Azure
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
F
IF:
6.1
论文数:
6.8K
被引数:
2.3W
机构
引用论文
Characterization of polymer matrix and low melting point solder for anisotropic conductive film各向异性导电膜用聚合物基体及低熔点焊料的表征
PREAI
分享收藏A lateralized alpha-band marker of the interference of exogenous attention over endogenous attention
PREAI
分享收藏Mean Square Convergent Non-Standard Numerical Schemes for Linear Random Differential Equations with Delay.png)
Mathematics
OAAI
MathematicsIF0
OAAI分享收藏
查看更多内容