.png){kind=icon}
返回
Multimodal query-guided object localization
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-023-15779-y.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recent studies have demonstrated the effectiveness of using hand-drawn sketches of objects as queries for one-shot object localization. However, hand-drawn crude sketches alone can be ambiguous for object localization, which could result in misidentification, e.g., a sketch of a laptop could be confused for a sofa. To overcome this, we propose a novel multimodal approach to object localization that combines sketch queries with linguistic category definitions, allowing for a better representation of visual and semantic cues. Our approach employs a cross-modal attention scheme that guides the region proposal network to obtain relevant proposals. Further, we propose an orthogonal projection-based proposal scoring technique that effectively ranks proposals with respect to the query. We evaluated our method using hand-drawn sketches from the 'Quick, Draw!' dataset and glosses from 'WordNet' as queries on the widely-used MS-COCO dataset, and achieve superior performance compared to related baselines in both open- and closed-set settings.
Keyword:
Sketch
Open-set object localization
Gloss
Cross-modal localization
Cross-modal attention
期刊
IF:
3
论文数:
1.9W
被引数:
3.2W
机构
引用论文
.png){kind=icon}
分享收藏
Conductivity Enhancement in Thin Silicon-on-Insulator Layer Embedding Artificial Dislocation Network
PREAI
分享收藏分享收藏分享收藏
查看更多内容