.png){kind=icon}
返回
Cross-modality interaction for few-shot multispectral object detection with semantic knowledge
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2024.106156.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multispectral object detection (MOD), which incorporates additional information from thermal images into object detection (OD) to robustly cope with complex illumination conditions, has garnered significant attention. However, existing MOD methods always demand a considerable amount of annotated data for training. Inspired by the concept of few -shot learning, we propose a novel task called few -shot multispectral object detection (FSMOD) that aims to accomplish MOD using only a few annotated data from each category. Specifically, we first design a cross -modality interaction (CMI) module, which leverages different attention mechanisms to interact with the information from visible and thermal modalities during backbone feature extraction. With the guidance of interaction process, the detector is able to extract modality -specific backbone features with better discrimination. To improve the few -shot learning ability of the detector, we also design a semantic prototype metric (SPM) loss that integrates semantic knowledge, i.e., word embeddings, into the optimization process of embedding space. Semantic knowledge provides stable category representation when visual information is insufficient. Extensive experiments on the customized FSMOD dataset demonstrate that the proposed method achieves state-of-the-art performance.
Keyword:
Few-shot learning
Object detection
Metric learning
Semantic knowledge
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
Specific Insulin‐Mediated Regulation of Glutamine Synthetase in Cultured Chick Astroglial Cells
PREAI
分享收藏
Thermal comfort, perceived air quality, and cognitive performance when personally controlled air movement is used by tropically acclimatized persons当热带适应的人使用个人控制的空气运动时，热舒适性，感知的空气质量和认知表现.png)
Indoor Air
OAAI
Indoor AirIF0
OAAI分享收藏Encoder deep interleaved network with multi-scale aggregation for RGB-D salient object detectionPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏Cross-modality interactive attention network for multispectral pedestrian detection用于多光谱行人检测的跨模态交互式注意网络INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
分享收藏
查看更多内容