.png){kind=icon}
.png){kind=icon}
Return
Towards Efficient Graph Processing in Geo-Distributed Data Centers
.png){kind=icon}
PRE
AI
DOI：10.1109/TPDS.2024.3453872.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Iterative graph processing is widely used as a significant paradigm for large-scale data analysis. In many global businesses of multinational enterprises, graph-structure data is usually geographically distributed in different regions to support low-latency services. Geo-distributed graph processing suffers from the Wide Area Networks (WANs) with scarce and heterogeneous bandwidth, thus essentially differs from traditional distributed graph processing. In this paper, we propose RAGraph, a Region-Aware framework for geo-distributed graph processing. At the core of RAGraph, we design a region-aware graph processing framework that allows advancing inefficient global updates locally and enables sensible coordination-free message interactions and flexible replaceable communication module. In terms of graph data preprocessing, RAGraph introduces a contribution-driven edge migration algorithm to effectively utilize network resources. RAGraph also contains an adaptive hierarchical message interaction engine to switch interaction modes adaptively based on network heterogeneity and fluctuation, and a discrepancy-aware message filtering strategy to filter important messages. Experimental results show that RAGraph can achieve an average speedup of 9.7x (up to 98x) and an average WAN cost reduction of 78.5$% (up to 97.3%) compared with state-of-the-art systems.
Keywords:
Data centers
Wide area networks
Bandwidth
Computational modeling
Iterative algorithms
Fluctuations
Measurement
Graph processing
geo-distributed data centers
heterogeneous network
Journal
IF:
6
Papers:
5.2K
Citations:
1.1W