.png){kind=icon}
Return
Efficient Distributed Mini-Batch GNN Training With Decentralized Batch Processing
.png){kind=icon}
PRE
AI
DOI：10.1109/tkde.2026.3721629.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Distributed mini-batch Graph Neural Network (GNN) training is a crucial technique for learning large-scale graphs. Existing training methods typically rely on a centralized batch processing pipeline, where a single worker manages the entire lifespan of a batch. Consequently, the communication volume is linearly proportional to the computation graph size, which grows exponentially with the number of GNN layers, leading to prohibitive communication costs. In this paper, we propose DEGNN, an efficient distributed GNN training framework that adopts a push computation to data strategy. By distributing the neighbor preparation workload to workers where the graph data resides, we design a decentralized batch processing pipeline in DEGNN that decouples the communication volume from the computation graph size. To realize this pipeline effectively, we propose LAMP, a Locality-Aware Multi-layer neighbor Preparation strategy. Specifically, LAMP constructs the aggregation skeleton via a locality-aware Computation Graph (CG) generation strategy. It then performs multi-layer aggregation to capture multi-hop neighborhood information without cross-worker communication, leveraging quantized historical embeddings to satisfy the recursive dependency of GNN layers. To further enhance end-to-end efficiency, we introduce a two-level computation graph fusion mechanism and an asynchronous pipeline to minimize computation overhead and hide communication latency. Empirical results demonstrate that DEGNN outperforms state-of-the-art distributed training systems, achieving up to 6.05× speedup while maintaining model accuracy.
Keywords:
Graph neural networks
distributed training
decentralized batch processing
Journal
IF:
10.4
Papers:
6.9K
Citations:
3.2W
Organization
Cited Papers
No cited papers available