.png){kind=icon}
.png){kind=icon}
Return
FLARE: Efficient Distributed Large-Scale Graph Neural Networks Training With Adaptive Latency-Aware Probabilistic Caching
.png){kind=icon}
PRE
AI
DOI：10.1109/TPDS.2026.3660379.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Since the emergence of Graph Neural Networks (GNNs), researchers have extensively investigated training on large-scale GNN training because of their success and wide usage in various domains including biological networks, finance, and recommendation systems. This work focuses on training large-scale distributed GNNs, where partitioning massive graphs across multiple machines creates remote communication overhead that becomes a major scalability bottleneck. We introduce a policy-driven caching mechanism that prioritizes node features and embeddings based on access frequency and cross-partition fetch cost, significantly minimizing communication overhead without sacrificing accuracy. Our policies are based on analysis of Node Affinities (NAFs) during multi-hop neighborhood sampling that extend substantially beyond the graph partition boundaries. Analyzing NAFs not only alleviates the communication bottleneck but also provides a systematic mechanism to manage in-memory data effectively, prioritizing GPU storage for node features with high fetch latency. We present FLARE, a system designed to handle partitioned feature data while leveraging the NAF-based caching policy. FLARE substantially reduces both communication overhead and training convergence time. Extensive experiments on benchmark datasets show that training FLARE on a three-layer GCN, GAT, and GraphSAGE across eight GPU machines achieves up to 12.04× (8.12× on average) speedup over DistDGLv2, demonstrating substantial performance gains compared to state-of-the-art methods.
Keywords:
Graph neural networks
distributed GNN training
node affinity
GPU caching
large-scale graphs
reinforcement learning
network latency
mini-batch sampling
parallel computing
Journal
IF:
6
Papers:
5.2K
Citations:
1.1W