.png){kind=icon}
返回
Data-Efficient Performance Modeling for Configurable Big Data Frameworks by Reducing Information Overlap Between Training Examples
.png){kind=icon}
PRE
AI
DOI：10.1016/j.bdr.2022.100358.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
To support the various analysis application of big data, big data processing frameworks are designed to be highly configurable. However, for common users, it is difficult to tailor the configurable frameworks to achieve optimal performance for every application. Recently, many automatic tuning methods are proposed to configure these frameworks. In detail, these methods firstly build a performance prediction model through sampling configurations randomly and measuring the corresponding performance. Then, they conduct heuristic search in the configuration space based on the performance prediction model. For most frameworks, it is too expensive to build the performance model since it needs to measure the performance of large amounts of configurations, which cause too much overhead on data collection. In this paper, we propose a novel data-efficient method to build the performance model with little impact on prediction accuracy. Compared to the traditional methods, the proposed method can reduce the overhead of data collection because it can train the performance model with much less training examples. Specifically, the proposed method can actively sample the important examples according to the dynamic requirement of the performance model during the iterative model updating. Hence, it can make full use of the collected informative data and train the performance model with much less training examples. To sample the important training examples, we employ several virtual performance model to estimate the importance of all candidate configurations efficiently. Experimental results show that our method needs less training examples than traditional methods with little impact on prediction accuracy.(c) 2022 Elsevier Inc. All rights reserved.
Keyword:
Big data framework
Highly configurable software
Performance model
Active sampling
期刊
IF:
4.2
论文数:
406
被引数:
1.1K
机构
暂无机构信息
引用论文
Efficient IoT-based sensor BIG Data collection-processing and analysis in smart buildings基于物联网的高效传感器大数据采集-智能建筑中的处理和分析
.png){kind=icon}
PREAI
分享收藏
没有更多内容
没有更多内容