.png){kind=icon}
Return
Contrast-Reconstruction Representation Learning for Self-Supervised Skeleton-Based Action Recognition
.png){kind=icon}
OA
AI
DOI：10.1109/TIP.2022.3207577.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Skeleton-based action recognition is widely used in varied areas, e.g., surveillance and human-machine interaction. Existing models are mainly learned in a supervised manner, thus heavily depending on large-scale labeled data, which could be infeasible when labels are prohibitively expensive. In this paper, we propose a novel Contrast-Reconstruction Representation Learning network (CRRL) that simultaneously captures postures and motion dynamics for unsupervised skeleton-based action recognition. It consists of three parts: Sequence Reconstructor (SER), Contrastive Motion Learner (CML), and Information Fuser (INF). SER learns representation from skeleton coordinate sequence via reconstruction. However the learned representation tends to focus on trivial postural coordinates and be hesitant in motion learning. To enhance the learning of motions, CML performs contrastive learning between the representation learned from coordinate sequences and additional velocity sequences, respectively. Finally, in the INF module, we explore varied strategies to combine SER and CML, and propose to couple postures and motions via a knowledge-distillation based fusion strategy which transfers the motion learning from CML to SER. Experimental results on several benchmarks, i.e., NTU RGB+D 60/120, PKU-MMD, CMU, and NW-UCLA, demonstrate the promise of the our method by outperforming state-of-the-art approaches.
Keywords:
Skeleton
Dynamics
Representation learning
Image reconstruction
Task analysis
Computational modeling
Visualization
Skeleton-based action recognition
unsupervised representation learning
contrastive learning
Journal
IF:
13.7
Papers:
1.0W
Citations:
8.4W
Organization
Cited Papers
ShareSaveShareSaveEvaluation of the Effect of Systolic Blood Pressure and Pulse Pressure on Cognitive Function: The Women's Health and Aging Study II.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAIShareSaveDependence of microstructural evolution on the geometric structure for serviced DZ125 turbine bladesOAAI
OAAIShareSaveShareSaveShareSave
View more