.png){kind=icon}
Return
Scalable parallel computing on clouds using Twister4Azure iterative MapReduce
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2012.05.027.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Recent advances in data-intensive computing for science discovery are fueling a dramatic growth in the use of data-intensive iterative computations. The utility computing model introduced by cloud computing, combined with the rich set of cloud infrastructure and storage services, offers a very attractive environment in which scientists can perform data analytics. The challenges to large-scale distributed computations on cloud environments demand innovative computational frameworks that are specifically tailored for cloud characteristics to easily and effectively harness the power of clouds. Twister4Azure is a distributed decentralized iterative MapReduce runtime for Windows Azure Cloud. Twister4Azure extends the familiar, easy-to-use MapReduce programming model with iterative extensions, enabling a fault-tolerance execution of a wide array of data mining and data analysis applications on the Azure cloud. Twister4Azure utilizes the scalable, distributed and highly available Azure cloud services as the underlying building blocks, and employs a decentralized control architecture that avoids single point failures. Twister4Azure optimizes the iterative computations using a multi-level caching of data, a cache aware decentralized task scheduling, hybrid tree-based data broadcasting and hybrid intermediate data communication. This paper presents the Twister4Azure iterative MapReduce runtime and a study of four real world data-intensive scientific applications implemented using Twister4Azure two iterative applications, Multi-Dimensional Scaling and KMeans Clustering; and two pleasingly parallel applications, BLAST+ sequence searching and SmithWaterman sequence alignment. Performance measurements show comparable or a factor of 2 to 4 better results than the traditional MapReduce runtimes deployed on up to 256 instances and for jobs with tens of thousands of tasks. We also study and present solutions to several factors that affect the performance of iterative MapReduce applications on Windows Azure Cloud, (c) 2012 Elsevier B.V. All rights reserved,
Keywords:
Iterative MapReduce
Cloud computing
HPC
Scientific applications
Azure
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
F
IF:
6.1
Papers:
6.9K
Citations:
2.3W
Organization
Cited Papers
Characterization of polymer matrix and low melting point solder for anisotropic conductive film
PREAI
ShareSaveShareSaveShareSaveShareSaveA lateralized alpha-band marker of the interference of exogenous attention over endogenous attention
PREAI
ShareSaveMean Square Convergent Non-Standard Numerical Schemes for Linear Random Differential Equations with Delay.png)
Mathematics
OAAI
MathematicsIF0
OAAIShareSave
View more