AdaNS: Adaptive negative sampling for unsupervised graph representation learning

doi:10.1016/j.patcog.2022.109266

AdaNS: Adaptive negative sampling for unsupervised graph representation learning

2023-04-01

4

PRE

AI

DOI：10.1016/j.patcog.2022.109266

原文链接

原文求助

摘要

En 中文

Recently, unsupervised graph representation learning has attracted considerable attention through ef-fectively encoding graph-structured data without semantic annotations. To accelerate its training, noise contrastive estimation (NCE) samples uniformly negative examples to fit an unnormalized graph model. However, this uniform sampling strategy may easily lead to slow convergence, even the vanishing gra-dient problem. In this paper, we theoretically show that sampling those hard negatives close to the cur-rent anchor can relieve the above difficulties. With this finding, we then propose an Adaptive Negative Sampling strategy, namely AdaNS, which efficiently samples the hard negatives from the mixing distribu-tion regarding the dimensional elements of the current node representation. Experiments show that our AdaNS sampling strategy applied on top of representative unsupervised models, e.g., DeepWalk, Graph -SAGE, can outperform the existing negative sampling strategies in the tasks of node classification and visualization. This also further demonstrates that sampling those hard negatives can bring performance improvements for learning the node representations.(c) 2022 Elsevier Ltd. All rights reserved.

Keyword:

Graph representation learning

Negative sampling

Noise contrastive estimation

AdaNS: Adaptive negative sampling for unsupervised graph representation learning

摘要

期刊

机构

引用论文