.png){kind=icon}
返回
Dynamic redirection of real-time data streams for elastic stream computing
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2020.05.021.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
An elastic stream computing system needs elastic adjustment of computing resource allocation and vertex parallelism to improve latency and throughput, which includes continuously or periodically scaling in/out the workload of computing nodes at runtime. Dynamic redirection can help with this elasticity issue by dynamically redirecting real-time data streams to computing resources. Due to the time-varying and unpredictable nature of real-time data streams, implementing redirection of data streams is challenging. Currently, the requirements of data streams redirection are not fully fulfilled, which directly affects the latency and throughput of stream computing systems. To bridge this gap, we proposed a dynamic redirection framework (called Dr-Stream) for elastic stream computing systems. This paper discussed the following aspects: (1) Investigating the dynamic redirection of real-time data streams, providing a general stream application model, a data stream model and a data stream grouping model, as well as formalizing the problem of load balancing optimization and data stream redirection. (2) Redirecting data streams among multiple instances of an operator at runtime by a lightweight load balancing strategy to improve the load balancing of a data center at the vertex level. Managing system states, especially the states of stateful operators by a logical ring-based strategy to improve accuracy. (3) Determining the number of instances for each operator, and deploying the instance(s) to computing nodes by a modified first-fit strategy at runtime. (4) Evaluating the fulfillment of low latency, high throughput, and load balancing objectives in a real-world distributed stream computing environment. Experimental results showed that Dr-Stream reduced the average system latency and load balancing of the data center by more than 20% and 15%, respectively. It also improved the average system stability by more than 15% and avoided over-utilization of computing nodes, as compared to the existing strategies in Storm. (C) 2020 Elsevier B.V. All rights reserved.
Keyword:
Data stream redirection
Stream computing
Elastic processing
Load balancing
Distributed system
Big data
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
F
IF:
6.1
论文数:
6.9K
被引数:
2.3W
机构
引用论文
分享收藏Twitter spam detection: Survey of new approaches and comparative study.png)
COMPUTERS & SECURITY
COMPUTERS & SECURITYIF5.4
PREAI
分享收藏
分享收藏
Energy and cost aware scheduling with batch processing for instance-intensive IoT workflows in clouds具有批处理功能的能源和成本感知调度，适用于云中的实例密集型IoT工作流
PREAI
分享收藏
查看更多内容