.png){kind=icon}
返回
Multi-modal visual tracking based on textual generation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.inffus.2024.102531.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-modal tracking has garnered significant attention due to its wide range of potential applications. Existing multi-modal tracking approaches typically merge data from different visual modalities on top of RGB tracking. However, focusing solely on the visual modality is insufficient due to the scarcity of tracking data. Inspired by the recent success of large models, this paper introduces a Multi-modal Visual Tracking Based on Textual Generation (MVTTG) approach to address the limitations of visual tracking, which lacks language information and overlooks semantic relationships between the target and the search area. To achieve this, we leverage large models to generate image descriptions, using these descriptions to provide complementary information about the target's appearance and movement. Furthermore, to enhance the consistency between visual and language modalities, we employ prompt learning and design a Visual-Language Interaction Prompt Manager (V-L PM) to facilitate collaborative learning between visual and language domains. Experiments conducted with MVTTG on multiple benchmark datasets confirm the effectiveness and potential of incorporating image descriptions in multi-modal visual tracking.
Keyword:
Multi-modal tracking
Image descriptions
Visual and language modalities
Prompt learning
期刊
IF:
15.5
论文数:
4.2K
被引数:
2.7W
机构
引用论文
分享收藏
Online Non-Negative Multi-Modality Feature Template Learning for RGB-Assisted Infrared Tracking.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏
查看更多内容