.png){kind=icon}
返回
Multi-scale motion contrastive learning for self-supervised skeleton-based action recognition
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-024-01463-0.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
People process things and express feelings through actions, action recognition has been able to be widely studied, yet under-explored. Traditional self-supervised skeleton-based action recognition focus on joint point features, ignoring the inherent semantic information of body structures at different scales. To address this problem, we propose a multi-scale Motion Contrastive Learning of Visual Representations (MsMCLR) model. The model utilizes the Multi-scale Motion Attention (MsM Attention) module to divide the skeletal features into three scale levels, extracting cross-frame and cross-node motion features from them. To obtain more motion patterns, a combination of strong data augmentation is used in the proposed model, which motivates the model to utilize more motion features. However, the feature sequences generated by strong data augmentation make it difficult to maintain identity of the original sequence. Hence, we introduce a dual distributional divergence minimization method, proposing a multi-scale motion loss function. It utilizes the embedding distribution of the ordinary augmentation branch to supervise the loss computation of the strong augmentation branch. Finally, the proposed method is evaluated on NTU RGB+D 60, NTU RGB+D 120, and PKU-MMD datasets. The accuracy of our method is 1.4-3.0% higher than the frontier models.
Keyword:
Contrastive learning
Multi-scale
Attention mechanism
Data augmentation
Human skeleton
期刊
IF:
3.1
论文数:
2.8K
被引数:
2.7K
机构
引用论文
Predictively encoded graph convolutional network for noise-robust skeleton-based action recognition基于预测编码的图卷积网络，用于噪声鲁棒的基于骨架的动作识别.png)
APPLIED INTELLIGENCE
OAAI
APPLIED INTELLIGENCEIF3.5
OAAI分享收藏Evaluation of the Effect of Systolic Blood Pressure and Pulse Pressure on Cognitive Function: The Women's Health and Aging Study IIPLoS ONEOAAI
PLoS ONEIF0
OAAI分享收藏Dependence of microstructural evolution on the geometric structure for serviced DZ125 turbine blades
OAAI
OAAI分享收藏分享收藏
查看更多内容