.png){kind=icon}
返回
Adaptive Latent Graph Representation Learning for Image-Text Matching
.png){kind=icon}
PRE
AI
DOI：10.1109/TIP.2022.3229631.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image-text matching is a challenging task due to the modality gap. Many recent methods focus on modeling entity relationships to learn a common embedding space of image and text. However, these methods suffer from distractions of entity relationships such as irrelevant visual regions in an image and noisy textual words in a text. In this paper, we propose an adaptive latent graph representation learning method to reduce the distractions of entity relationships for image-text matching. Specifically, we use an improved graph variational autoencoder to separate the distracting factors and latent factor of relationships and jointly learn latent textual graph representations, latent visual graph representations, and a visual-textual graph embedding space. We also introduce an adaptive cross-attention mechanism to perform feature attending on the latent graph representations across images and texts, thus further narrowing the modality gap to boost the matching performance. Extensive experiments on two public datasets, Flickr30K and COCO, show the effectiveness of our method.
Keyword:
Image-text matching
latent representation learning
graph variational autoencoder
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
.png){kind=icon}
Cross-Domain Image Captioning via Cross-Modal Retrieval and Model Adaptation通过跨模态检索和模型自适应实现跨域图像字幕
PREAI
分享收藏Deep multimodal learning for cross-modal retrieval: One model for all tasks用于跨模态检索的深度多模态学习: 一个用于所有任务的模型
OAAI
OAAI分享收藏
查看更多内容