.png){kind=icon}
返回
Prototype local-global alignment network for image-text retrieval
.png){kind=icon}
PRE
AI
DOI：10.1007/s13735-022-00258-1.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image-text retrieval is a challenging task due to the requirement of thorough multimodal understanding and precise inter-modality relationship discovery. However, most previous approaches resort to doing global image-text alignment and neglect fine-grained correspondence. Although some works explore local region-word alignment, they usually suffer from a heavy computing burden. In this paper, we propose a prototype local-global alignment (PLGA) network for image-text retrieval by jointly performing the fine-grained local alignment and high-level global alignment. Specifically, our PLGA contains two key components: a prototype-based local alignment module and a multi-scale global alignment module. The former enables efficient fine-grained local matching by combining region-prototype alignment and word-prototype alignment, and the latter helps perceive hierarchical global semantics by exploring multi-scale global correlations between the image and text. Overall, the local and global alignment modules can boost their performances for each other via the unified model. Quantitative and qualitative experimental results on Flickr30K and MS-COCO benchmarks demonstrate that our proposed approach performs favorably against state-of-the-art methods.
Keyword:
Image-text retrieval
Local alignment
Global alignment
Prototype
期刊
IF:
2.9
论文数:
278
被引数:
866
机构
引用论文
.png){kind=icon}
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏分享收藏分享收藏603 VEDOLIZUMAB PREVENTS POSTOPERATIVE REUCURRENCE IN CROHN'S DISEASE: RESULTS OF THE REPREVIO TRIAL
PREAI
分享收藏
查看更多内容