.png){kind=icon}
返回
Contrastive predictive coding with transformer for video representation learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2021.11.031.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This paper presents a novel framework of self-supervised learning for video representation. Inspired by Contrastive Predictive Coding and Self-attention, we make the following contributions: First, we propose the Contrastive Predictive Coding with Transformer (CPCTR) framework for video representation learning in a self-supervised fashion. Second, we introduce the Transformer architecture to CPCTR to capture long-range spatio-temporal dependencies in order to facilitate the learning of slow features in video, and we conduct analysis of Transformer in our model to show its effectiveness. Finally, we evaluate our model by first training on the UCF101 dataset with self-supervised learning, and then fine-tuning on downstream video classification tasks. Using RGB only video data, we achieve state-of-the-art self-supervised performance on both UCF101 (Top1 accuracy of 99.3%) and HMDB51 (Top1 accuracy of 82.4%), we show that CPCTR even outperforms fully supervised methods on the two datasets. The code is available at https://github.com/yliu1229/CPCTR. (C) 2021 Elsevier B.V. All rights reserved.
Keyword:
Contrastive Learning
Self-Attention
Video Representation
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
分享收藏
分享收藏
没有更多内容
没有更多内容