.png){kind=icon}
返回
Proformer: a scalable graph transformer with linear complexity
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-06065-x.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Since existing GNN methods use a fixed input graph structure for messages passing, they cannot solve the problems of heterogeneity, over-squashing, long-range dependencies, and graph incompleteness. The all-pair message passing scheme is an effective means to address the above issues. However, owing to the quadratic complexity problem of self-attention used in the all-pair message passing scheme, it is not possible to simultaneously guarantee the scalability and accuracy of the algorithm on large-scale graph datasets. In this paper, we propose Proformer, which uses multilayer dilation convolution to project the key and value in self-attention and uses a focused function to further enhance the model representation and reduce the computational complexity of the all-pair message passing scheme from quadratic to linear. The experimental results show that Proformer performs very well in tasks such as nodes, images, and text. Additionally, when scaled to large-scale graph datasets, it is able to effectively reduce the inference time and GPU memory utilization while guaranteeing the algorithm's accuracy. On OGB-Proteins, it not only improves the ROC-AUC by 3.2% but also conserves 27.8% of the GPU memory.
Keyword:
GNN
Self-attention
Quadratic complexity
Dilation convolution
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
引用论文
Correlation between vibrational modes and structural characteristics of Ba[(Zn1−xMgx)1/3Ta2/3]O3 solid solutions.png)
CrystEngComm
CrystEngCommIF0
PREAI
分享收藏KNN-GNN: A powerful graph neural network enhanced by aggregating K-nearest neighbors in common subspaceKnn-gnn: 一个强大的图形神经网络，通过在公共子空间中聚合K近邻来增强
PREAI
分享收藏
On the Production Of Energy From Sea Waves By a Rotating Pendulum: A Preliminary Experimental Study
PREAI
分享收藏
查看更多内容