.png){kind=icon}
返回
Semi-supervised constrained graph convolutional network for cross-modal retrieval
.png){kind=icon}
PRE
AI
DOI：10.1016/j.compeleceng.2022.107994.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Exploiting relationship among samples in cross-modal data plays a key role in the task of cross modal retrieval, but most of existing methods only extract the correlation from pairwise samples and ignore the relations of unpaired samples. Some graph regularization methods proposed a reasonable paradigm to exploit the correlation from multiple samples. However, limited by the traditional framework, the performance has much room to improve. Moreover, although some existing DNN-based methods achieve excellent performance, the requirement of massive labeled data is also a shortcoming. In this paper, we propose a novel semi-supervised method, named Semi-supervised Constrained Graph Convolutional Network (SCGCN), which adopts graph convolutional network to exploit correlation from batch samples of data with different modalities. For reducing the requirement of labeled data, we design a two stage training procedure: deep supervised learning stage and unsupervised learning stage. In deep supervised learning stage, we integrate two DNN-based semantic encoding networks and a shared classifier into Deep Cross-modal Semantic Encoding (DCSE) module which is trained by supervised learning with labeled data. From DCSE module, we learn a temporary modality-invariant space where the semantic embeddings of samples with different modalities are modality-invariant, and we also learn a classifier which can generate predicted label from the unlabeled data. In unsupervised learning stage, for fully exploiting the correlation from cross-modal data, we design a Constrained Graph Convolutional Network (CGCN) module which utilizes GCN to exploit the correlation and adopts both intra-modal discriminative loss and inter-modal pairwise similar loss to ensure the generated common representation modality-invariant and semantical discriminative. We perform extensive experiments on four conventional datasets and a large scale dataset to demonstrate the effectiveness of proposed approach.
Keyword:
Cross-modal retrieval
Semi-supervised learning
Deep neural network
Graph convolutional network
期刊
C
IF:
4.9
论文数:
6.7K
被引数:
1.3W
机构
暂无机构信息
引用论文
The effect of decomposition of beta-phase Zr-20 at% Nb on hydrogen partitioning with alpha-zirconium
PREAI
分享收藏Solution Structure of Monomeric Peptide YY Supports the Functional Significance of the PP-Fold,.png)
Biochemistry
BiochemistryIF0
PREAI
分享收藏The effect of a novel extracorporeal cytokine hemoadsorption device on IL-6 elimination in septic patients: A randomized controlled trialPLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏CCL: Cross-modal Correlation Learning With Multigrained Fusion by Hierarchical NetworkCCL: 分层网络多粒度融合的跨模态相关学习OAAI
OAAI分享收藏Construction of a Hierarchical Feature Enhancement Network and Its Application in Fault Recognition
PREAI
分享收藏
查看更多内容