.png){kind=icon}
返回
Deadline-Aware Cost Optimization for Spark
.png){kind=icon}
PRE
AI
DOI：10.1109/TBDATA.2019.2908188.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We present OptEx, a closed-form model of job execution on Apache Spark, a popular parallel processing engine. To the best of our knowledge, OptEx is the first work that analytically models job completion time on Spark. The model can be used to estimate the completion time of a given Spark job on a cloud, with respect to the size of the input dataset, the number of iterations, and the number of nodes comprising the underlying cluster. Experimental results demonstrate that OptEx yields a mean relative error of 6 percent in estimating the job completion time. Furthermore, the model can be applied for estimating the cost-optimal cluster composition for running a given Spark job on a cloud under a completion deadline specified in the SLO (i.e., Service Level Objective). We show experimentally that OptEx is able to correctly estimate the required cluster composition for running a given Spark job under a given SLO deadline with an accuracy of 98 percent. We also provide a tool which can classify Spark jobs into job categories based on bisimilarity analysis on lineage graphs collected from the given jobs.
Keyword:
Sparks
Cloud computing
Virtual machining
Big Data
Tools
Parallel processing
Input variables
Distributed systems
parallel processing
distributed file systems
middleware
performance evaluation
reliability
availability
and serviceability
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
I
IF:
5.7
论文数:
892
被引数:
3.0K
机构
引用论文
Synthesis, in-situ membrane preparation, and good gas permselectivity of insoluble poly(substituted acetylene)s loosely cross-linked with short and soft siloxane and silanol linkages.png)
Polymer
PolymerIF0
PREAI
分享收藏
查看更多内容