.png){kind=icon}
返回
Self-supervised learning for RGB-D object tracking
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110543.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, there has been a growing interest in RGB-D object tracking thanks to its promising performance achieved by combining visual information with auxiliary depth cues. However, the limited volume of annotated RGB-D tracking data for offline training has hindered the development of a dedicated end -to -end RGB-D tracker design. Consequently, the current state-of-the-art RGB-D trackers mainly rely on the visual branch to support the appearance modelling, with the depth map utilised for elementary information fusion or failure reasoning of online tracking. Despite the achieved progress, the current paradigms for RGB-D tracking have not fully harnessed the inherent potential of depth information, nor fully exploited the synergy of vision -depth information. Considering the availability of ample unlabelled RGB-D data and the advancement in self supervised learning, we address the problem of self -supervised learning for RGB-D object tracking. Specifically, an RGB-D backbone network is trained on unlabelled RGB-D datasets using masked image modelling. To train the network, the masking mechanism creates a selective occlusion of the input visible image to force the corresponding aligned depth map to help with discerning and learning vision -depth cues for the reconstruction of the masked visible image. As a result, the pre -trained backbone network is capable of cooperating with crucial visual and depth features of the diverse objects and background in the RGB-D image. The intermediate RGB-D features output by the pre -trained network can effectively be used for object tracking. We thus embed the pre -trained RGB-D network into a transformer -based tracking framework for stable tracking. Comprehensive experiments and the analysis of the results obtained on several RGB-D tracking datasets demonstrate the effectiveness and superiority of the proposed RGB-D self -supervised learning framework and the following tracking approach.
Keyword:
RGB-D tracking
Self-supervised learning
Masked image modelling
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Determining Critical Success Factors of Urban Renewal Projects: Multiple Integrated Approach确定城市更新项目的关键成功因素: 多重综合方法
.png){kind=icon}
PREAI
分享收藏
分享收藏Geriatric Assessments and Medication Therapy Management Approaches in Elderly Oncology Patients
PREAI
分享收藏
查看更多内容