.png){kind=icon}
返回
Progressive Language-Customized Visual Feature Learning for One-Stage Visual Grounding
.png){kind=icon}
PRE
AI
DOI：10.1109/TIP.2022.3181516.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Visual grounding is a task to localize an object described by a sentence in an image. Conventional visual grounding methods extract visual and linguistic features isolatedly and then perform cross-modal interaction in a post-fusion manner. We argue that this post-fusion mechanism does not fully utilize the information in two modalities. Instead, it is more desired to perform cross-modal interaction during the extraction process of the visual and linguistic feature. In this paper, we propose a language-customized visual feature learning mechanism where linguistic information guides the extraction of visual feature from the very beginning. We instantiate the mechanism as a one-stage framework named Progressive Language-customized Visual feature learning (PLV). Our proposed PLV consists of a Progressive Language-customized Visual Encoder (PLVE) and a grounding module. We customize the visual feature with linguistic guidance at each stage of the PLVE by Channel-wise Language-guided Interaction Modules (CLIM). Our proposed PLV outperforms conventional state-of-the-art methods with large margins across five visual grounding datasets without pre-training on object detection datasets, while achieving real-time speed. The source code is available in the supplementary material.
Keyword:
Visualization
Feature extraction
Grounding
Linguistics
Task analysis
Detectors
Representation learning
Visual grounding
referring expression comprehension
visual linguistic understanding
cross-modal fusion
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
Study on Adhesive Strength between Ge2Sb2Te5Film and Electrodes for Phase Change Memory Application
PREAI
分享收藏分享收藏Substrate specificity and inhibitor analyses of human steroid 5β-reductase (AKR1D1)人类类固醇5 β-还原酶 (AKR1D1) 的底物特异性和抑制剂分析.png)
Steroids
OAAI
SteroidsIF0
OAAI分享收藏A retrospective 5-year review of rubella in South Africa prior to the introduction of a rubella-containing vaccinePLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏Communication and reproductive behaviour in North American Jerusalem crickets (
Stenopelmatus
) (Orthoptera: Stenopelmatidae).北美耶路撒冷蟋蟀的交流和生殖行为 (
Stenopelmatus
) (直翅目: Stenopelmatidae)。
PREAI
分享收藏
查看更多内容