.png){kind=icon}
返回
Distributed Graph Neural Network Training: A Survey
.png){kind=icon}
OA
AI
DOI：10.1145/3648358.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Graph neural networks (GNNs) are a type of deep learning models that are trained on graphs and have been successfully applied in various domains. Despite the effectiveness of GNNs, it is still challenging for GNNs to efficiently scale to large graphs. As a remedy, distributed computing becomes a promising solution of training large-scale GNNs, since it is able to provide abundant computing resources. However, the dependency of graph structure increases the difficulty of achieving high-efficiency distributed GNN training, which suffers from the massive communication and workload imbalance. In recent years, many efforts have been made on distributed GNN training, and an array of training algorithms and systems have been proposed. Yet, there is a lack of systematic review of the optimization techniques for the distributed execution of GNN training. In this survey, we analyze three major challenges in distributed GNN training: massive feature communication, the loss of model accuracy, and workload imbalance. Then, we introduce a new taxonomy for the optimization techniques in distributed GNN training that address the above challenges. The new taxonomy classifies existing techniques into four categories: GNN data partition, GNN batch generation, GNN execution model, and GNN communication protocol. We carefully discuss the techniques in each category. In the conclusion, we summarize existing distributed GNN systems for multi-graphics processing units (GPUs), GPU-clusters and central processing unit (CPU)-clusters, respectively, and present a discussion about the future direction of distributed GNN training.
Keyword:
Surveys and overviews
distributed GNN training
graph data management
communication optimization
distributed GNN systems
期刊
IF:
28
论文数:
2.4K
被引数:
3.5W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏Few-Shot Relation Prediction of Knowledge Graph via Convolutional Neural Network with Self-Attention基于自注意的卷积神经网络知识图谱少镜头关系预测OAAI
OAAI分享收藏
查看更多内容