.png){kind=icon}
返回
GCL: Contrastive learning instead of graph convolution for node classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2023.126491.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Contrastive learning as an effective representation learning technique has attracted tremendous attention due to its general success in downstream tasks. However, the theoretical explanations and quantitative experimental analyses of its generalization ability are still limited. These issues are pivotal yet challenging for improving both the interpretability and performance of contrastive learning. To address these issues, we first re-examine the least squares bias-variance decomposition and successfully derive GCL, a novel bias-variance decomposition with two optional generalized biases and one generalized variance. GCL is shown to be extendable to common contrastive learning models so that it can be utilized as a unified contrastive learning framework. Meanwhile, a surprising finding that the gradient descent of contrastive loss concerning feature representation is closely related to the message passing mechanism (graph convolution) of Graph Neural Networks (GNNs). The contrastive learning model called GCP is then proposed as a convincing implementation of GCL. GCP has a pure MLP-based structure and employs a conventional cross-entropy to reduce the bias between predictions and ground truth labels, two optional contrastive losses to optimize the variance of the model. Finally, extensive experiments demonstrate that the two biases proposed by GCL have their own merits; GCP achieves comparable or even better performance than GNNs in a more efficient and robust manner, its bias and variance meet the bias-variance tradeoff to some extent.(c) 2023 Elsevier B.V. All rights reserved.
Keyword:
Contrastive learning
Graph neural network
Bias-variance tradeoff
Generalization error
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
分享收藏分享收藏Contrastive Graph Convolutional Networks with adaptive augmentation for text classification基于自适应增强的对比图卷积网络文本分类
PREAI
分享收藏
Ultrasound-Mediated Atom Transfer Radical Polymerization for Polymer-Grafted Magnetic Nanoparticles with Controlled Packing Density超声辅助原子转移自由基聚合制备具有可控堆积密度的聚合物接枝磁性纳米粒子
OAAI
OAAI分享收藏A Quantitative Relationship between Oxidation Index and Chalcopyrite Flotation Recovery.png)
MineralsOAAI
MineralsIF0
OAAI分享收藏A Contrastive learning-based Task Adaptation model for few-shot intent recognition基于对比学习的少镜头意图识别任务适应模型
PREAI
分享收藏
查看更多内容