Structure-Aware DropEdge Toward Deep Graph Convolutional Networks

doi:10.1109/TNNLS.2023.3288484

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks

2024-11-01

0

OA

AI

J

Jiaqi Han

黄

黄文炳 (Wenbing Huang) *

DOI：10.1109/TNNLS.2023.3288484

Original

Original request for help

Save

Abstract

En 中文

It has been discovered that graph convolutional networks (GCNs) encounter a remarkable drop in performance when multiple layers are piled up. The main factor that accounts for why deep GCNs fail lies in oversmoothing, which isolates the network output from the input with the increase of network depth, weakening expressivity and trainability. In this article, we start by investigating refined measures upon DropEdge-an existing simple yet effective technique to relieve oversmoothing. We term our method as DropEdge $++$ for its two structure-aware samplers in contrast to DropEdge: layer-dependent (LD) sampler and feature-dependent (FD) sampler. Regarding the LD sampler, we interestingly find that increasingly sampling edges from the bottom layer yields superior performance than the decreasing counterpart as well as DropEdge. We theoretically reveal this phenomenon with mean-edge-number (MEN), a metric closely related to oversmoothing. For the FD sampler, we associate the edge sampling probability with the feature similarity of node pairs and prove that it further correlates the convergence subspace of the output layer with the input features. Extensive experiments on several node classification benchmarks, including both full-and semi-supervised tasks, illustrate the efficacy of DropEdge $++$ and its compatibility with a variety of backbones by achieving generally better performance over DropEdge and the no-drop version.

Keywords:

DropEdge plus plus

feature-dependent (FD) sampler

graph convolutional network (GCN)

layer-dependent (LD) sam-pler

mean-edge-number (MEN)

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks

Abstract

Journal

Organization