Learning Language Prompt for Vision-Language Tracking

doi:10.1109/TCSVT.2025.3557053

PRE

Chengao Zong

DOI：10.1109/TCSVT.2025.3557053

原文链接

原文求助

摘要

En 中文

视觉语言目标跟踪整合了先进的语言信息，增强了其在复杂场景中的鲁棒性和准确性。然而，当前方法受限于缺乏充足的视觉语言数据，导致模型难以学习泛化知识。为缓解此问题，我们提出了一种新的基于提示的视觉语言跟踪框架，命名为ProVLT。该框架将语言信息视为预训练视觉跟踪模型的提示，从而利用大量跟踪数据中的知识。实验表明，ProVLT在仅训练少量参数（约为模态参数的29%）的情况下实现了有竞争力的性能。例如，ProVLT在TNL2K基准上达到了59.8%的AUC。此外，我们为五个主流纯视觉跟踪基准添加了语言注释，并发现语言信息的加入始终提升了跟踪性能。在这些基准上，语言信息使跟踪性能相比纯视觉跟踪器平均提升了2.9%。我们将向社区发布代码、模型和基准。

Keyword:

Vision-language tracking

prompt learning

language annotation

期刊

IEEE Transactions on Circuits and Systems for Video Technology

IF:

11.1

论文数:

846

被引数:

3.1W

机构

Dalian University of Technology

学者数:

6.0W

论文数: 4.4W

被引数: 5.5W

引用论文

AiATrack: Attention in Attention for Transformer Visual Tracking

Lecture Notes in Computer Science

IF0

2022-10-23

PREAI

Shenyuan Gao; Chunluan Zhou; Chao Ma; Xinggang Wang; Junsong Yuan

The Ninth Visual Object Tracking VOT2021 Challenge Results

2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)

IF0

2021-10-01

OAAI

Matej Kristan; Jiri Matas; Ales Leonardis; Michael Felsberg; Roman Pflugfelder; Joni-Kristian Kamarainen; Hyung Jin Chang; Martin Danelljan; Luka Cehovin Zajc; Alan Lukezic; Ondrej Drbohlav; Jani Kapyla; Gustav Hager; Song Yan; Jinyu Yang; Zhongqun Zhang; Gustavo Fernandez; Mohamed Abdelpakey; Goutam Bhat; Llukman Cerkezi; Hakan Cevikalp; Shengyong Chen; Xin Chen; Miao Cheng; Ziyi Cheng; Yu-Chen Chiu; Ozgun Cirakman; Yutao Cui; Kenan Dai; Mohana Murali Dasari; Qili Deng; Xingping Dong; Daniel K. Du; Matteo Dunnhofer; Zhen-Hua Feng; Zhiyong Feng; Zhihong Fu; Shiming Ge; Rama Krishna Gorthi; Yuzhang Gu; Bilge Gunsel; Qing Guo; Filiz Gurkan; Wencheng Han; Yanyan Huang; Felix Jaremo Lawin; Shang-Jhih Jhang; Rongrong Ji; Cheng Jiang; Yingjie Jiang; Felix Juefei-Xu; Yin Jun; Xiao Ke; Fahad Shahbaz Khan; Byeong Hak Kim; Josef Kittler; Xiangyuan Lan; Jun Ha Lee; Bastian Leibe; Hui Li; Jianhua Li; Xianxian Li; Yuezhou Li; Bo Liu; Chang Liu; Jingen Liu; Li Liu; Qingjie Liu; Huchuan Lu; Wei Lu; Jonathon Luiten; Jie Ma; Ziang Ma; Niki Martinel; Christoph Mayer; Alireza Memarmoghadam; Christian Micheloni; Yuzhen Niu; Danda Paudel; Houwen Peng; Shoumeng Qiu; Aravindh Rajiv; Muhammad Rana; Andreas Robinson; Hasan Saribas; Ling Shao; Mohamed Shehata; Furao Shen; Jianbing Shen; Kristian Simonato; Xiaoning Song; Zhangyong Tang; Radu Timofte; Philip Torr; Chi-Yi Tsai; Bedirhan Uzun; Luc Van Gool; Paul Voigtlaender; Dong Wang; Guangting Wang; Liangliang Wang; Lijun Wang; Limin Wang; Linyuan Wang; Yong Wang; Yunhong Wang; Chenyan Wu; Gangshan Wu; Xiao-Jun Wu; Fei Xie; Tianyang Xu; Xiang Xu; Wanli Xue; Bin Yan; Wankou Yang; Xiaoyun Yang; Yu Ye; Jun Yin; Chengwei Zhang; Chunhui Zhang; Haitao Zhang; Kaihua Zhang; Kangkai Zhang; Xiaohan Zhang; Xiaolin Zhang; Xinyu Zhang; Zhibin Zhang; Shaochuan Zhao; Ming Zhen; Bineng Zhong; Jiawen Zhu; Xue-Feng Zhu

Autoregressive Visual Tracking

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2023-06-01

PREAI

Xing Wei; Yifan Bai; Yongchao Zheng; Dahu Shi; Yihong Gong

Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking

IEEE TRANSACTIONS ON IMAGE PROCESSING

IF13.7

2021-01-01

103

OAAI

Zhang, Pengyu; Zhao, Jie; Bo, Chunjuan; Wang, Dong; Lu, Huchuan; Yang, Xiaoyun

Transformer vision-language tracking via proxy token guided cross-modal fusion Transformer vision-通过代理令牌引导的跨模态融合进行语言跟踪

PATTERN RECOGNITION LETTERS

IF3.3

2023-04-01

PREAI

Zhao, Haojie; Wang, Xiao; Wang, Dong; Lu, Huchuan; Ruan, Xiang

Natural Language Guided Attention Mixer for Object Tracking

2023 4th International Conference on Information Science, Parallel and Distributed Systems (ISPDS)

IF0

2023-07-14

PREAI

Lu,Qingyang; Yuan,Guanglin; Li,Congli; Zhu,Hong; Qin,Xiaoyan

The Eighth Visual Object Tracking VOT2020 Challenge Results

Lecture Notes in Computer Science

IF0

2021-01-31

PREAI

Matej Kristan; Aleš Leonardis; Jiří Matas; Michael Felsberg; Roman Pflugfelder; Joni-Kristian Kämäräinen; Martin Danelljan; Luka Čehovin Zajc; Alan Lukežič; Ondrej Drbohlav; Linbo He; Yushan Zhang; Song Yan; Jinyu Yang; Gustavo Fernández; Alexander Hauptmann; Alireza Memarmoghadam; Álvaro García-Martín; Andreas Robinson; Anton Varfolomieiev; Awet Haileslassie Gebrehiwot; Bedirhan Uzun; Bin Yan; Bing Li; Chen Qian; Chi-Yi Tsai; Christian Micheloni; Dong Wang; Fei Wang; Fei Xie; Felix Jaremo Lawin; Fredrik Gustafsson; Gian Luca Foresti; Goutam Bhat; Guangqi Chen; Haibin Ling; Haitao Zhang; Hakan Cevikalp; Haojie Zhao; Haoran Bai; Hari Chandana Kuchibhotla; Hasan Saribas; Heng Fan; Hossein Ghanei-Yakhdan; Houqiang Li; Houwen Peng; Huchuan Lu; Hui Li; Javad Khaghani; Jesus Bescos; Jianhua Li; Jianlong Fu; Jiaqian Yu; Jingtao Xu; Josef Kittler; Jun Yin; Junhyun Lee; Kaicheng Yu; Kaiwen Liu; Kang Yang; Kenan Dai; Li Cheng; Li Zhang; Lijun Wang; Linyuan Wang; Luc Van Gool; Luca Bertinetto; Matteo Dunnhofer; Miao Cheng; Mohana Murali Dasari; Ning Wang; Ning Wang; Pengyu Zhang; Philip H. S. Torr; Qiang Wang; Radu Timofte; Rama Krishna Sai Gorthi; Seokeon Choi; Seyed Mojtaba Marvasti-Zadeh; Shaochuan Zhao; Shohreh Kasaei; Shoumeng Qiu; Shuhao Chen; Thomas B. Schön; Tianyang Xu; Wei Lu; Weiming Hu; Wengang Zhou; Xi Qiu; Xiao Ke; Xiao-Jun Wu; Xiaolin Zhang; Xiaoyun Yang; Xuefeng Zhu; Yingjie Jiang; Yingming Wang; Yiwei Chen; Yu Ye; Yuezhou Li; Yuncon Yao; Yunsung Lee; Yuzhang Gu; Zezhou Wang; Zhangyong Tang; Zhen-Hua Feng; Zhijun Mai; Zhipeng Zhang; Zhirong Wu; Ziang Ma

Tracking by Natural Language Specification

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2017-07-01

OAAI

Zhenyang Li; Ran Tao; Efstratios Gavves; Cees G. M. Snoek; Arnold W. M. Smeulders

Toward Modalities Correlation for RGB-T Tracking

IEEE Transactions on Circuits and Systems for Video Technology

IF0

2024-10-01

PREAI

Xiantao Hu; Bineng Zhong; Qihua Liang; Shengping Zhang; Ning Li; Xianxian Li