.png){kind=icon}
Return
SDN-enabled Resource Provisioning Framework for Geo-Distributed Streaming Analytics
.png){kind=icon}
OA
AI
DOI：10.1145/3571158.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Geographically distributed (geo-distributed) datacenters for stream data processing typically comprise multiple edges and core datacenters connected through Wide-Area Network (WAN) with a master node responsible for allocating tasks to worker nodes. Since WAN links significantly impact the performance of distributed task execution, the existing task assignment approach is unsuitable for distributed stream data processing with low latency and high throughput demand. In this paper, we propose SAFA, a resource provisioning framework using the Software-Defined Networking (SDN) concept with an SDN controller responsible for monitoring the WAN, selecting an appropriate subset of worker nodes, and assigning tasks to the designated worker nodes. We implemented the data plane of the framework in P4 and the control plane components in Python. We tested the performance of the proposed system on Apache Spark, Apache Storm, and Apache Flink using the Yahoo! streaming benchmark on a set of custom topologies. The results of the experiments validate that the proposed approach is viable for distributed stream processing and confirm that it can improve at least 1.64x the processing time of incoming events of the current stream processing systems.
Keywords:
Cluster manager
geo-distributed stream analytics
stream processing
Software-Defined Networking (SDN)
Journal
IF:
4.1
Papers:
896
Citations:
1.9K
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSaveSDN-based and multitenant-aware resource provisioning mechanism for cloud-based big data streaming
PREAI
ShareSave
View more