DropNaE: Alleviating irregularity for large-scale graph representation learning

doi:10.1016/j.neunet.2024.106930

DropNaE: Alleviating irregularity for large-scale graph representation learning

2025-03-01

0

PRE

AI

DOI：10.1016/j.neunet.2024.106930

Original

Original request for help

Save

Abstract

En 中文

Large-scale graphs are prevalent in various real-world scenarios and can be effectively processed using Graph Neural Networks (GNNs) on GPUs to derive meaningful representations. However, the inherent irregularity found in real-world graphs poses challenges for leveraging the single-instruction multiple-data execution mode of GPUs, leading to inefficiencies in GNN training. In this paper, we try to alleviate this irregularity at its origin-the irregular graph data itself. To this end, we propose DropNaE to alleviate the irregularity in large-scale graphs by conditionally dropping nodes and edges before GNN training. Specifically, we first present a metric to quantify the neighbor heterophily of all nodes in a graph. Then, we propose DropNaE containing two variants to transform the irregular degree distribution of the large-scale graph to a uniform one, based on the proposed metric. Experiments show that DropNaE is highly compatible and can be integrated into popular GNNs to promote both training efficiency and accuracy of used GNNs. DropNaE is offline performed and requires no online computing resources, benefiting the state-of-the-art GNNs in the present and future to a significant extent.

Keywords:

Algorithms on graph representation learning

Efficient large-scale graph representation

learning

Irregularity

DropNaE: Alleviating irregularity for large-scale graph representation learning

Abstract

Journal

Organization