.png){kind=icon}
返回
Continuous frame motion sensitive self-supervised collaborative network for video representation learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.aei.2023.101941.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Motion, as a feature of video that changes in temporal sequences, is crucial to visual understanding. The powerful video representation and extraction models are typically able to focus attention on motion features in challenging dynamic environments to complete more complex video understanding tasks. However, previous approaches discriminate mainly based on similar features in the spatial or temporal domain, ignoring the interdependence of consecutive video frames. In this paper, we propose the motion sensitive self-supervised collaborative network, a video representation learning framework that exploits a pretext task to assist feature comparison and strengthen the spatiotemporal discrimination power of the model. Specifically, we first propose the motion-aware module, which extracts consecutive motion features from the spatial regions by frame difference. The global-local contrastive module is then introduced, with context and enhanced video snippets being defined as appropriate positive samples for a broader feature similarity comparison. Finally, we introduce the snippet operation prediction module, which further assists contrastive learning to obtain more reliable global semantics by sensing changes in continuous frame features. Experimental results demonstrate that our work can effectively extract robust motion features and achieve competitive performance compared with other state-of-the-art self-supervised methods on downstream action recognition and video retrieval tasks.
Keyword:
Self-supervised representation learning
Pretext task
Global-local contrastive learning
Action recognition
Video retrieval
期刊
IF:
9.9
论文数:
4.4K
被引数:
1.7W
机构
引用论文
Diagnostic ability of confocal near-infrared reflectance fundus imaging to detect retrograde microcystic maculopathy from chiasm compression. A comparative study with OCT findings.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAI分享收藏
Strain-tunable electronic, elastic, and optical properties of CaI2monolayer: first-principles study
PREAI
分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏分享收藏
查看更多内容