.png){kind=icon}
返回
Improving self-supervised action recognition from extremely augmented skeleton sequences
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2024.110333.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Self-supervised contrastive learning has been widely applied to skeleton-based action recognition due to its ability to learn discriminative features. However, directly applying the existing contrastive learning framework for 3D skeleton learning is limited by the well-designed augmentations and the simple multi-stream decisionlevel fusion. To deal with these drawbacks, we propose a three-stream contrastive learning framework utilizing abundant information mining for self-supervised action representation (3s-AimCLR++). For single-stream contrastive learning, extreme augmentation is first proposed to generate more movement patterns, which can introduce more movement patterns to improve the universality of the learned representations. Since directly using extreme augmentation can barely boost the performance due to the drastic changes in original identity, the Distributional Divergence Minimization (DDM) loss is proposed to utilize the extreme augmentation more gently. Moreover, the Single-Stream Nearest Neighbors Mining (SNNM) is proposed to expand positive samples to make the learning process more reasonable. For multi-stream, existing methods simply ensemble the results. Yet, considering the complementarity of information between different streams, we propose Multi-Stream Aggregation and Interaction (MSAI) strategy to better fuse multi-stream information. Extensive experiments on NTU-60, NTU-120, and PKU-MMD datasets have verified that our 3s-AimCLR++ can significantly perform favorably against state -of -the -art methods under a variety of evaluation protocols. The code and models are available at https://github.com/Levigty/AimCLR-v2.
Keyword:
Self-supervised skeleton-based action
recognition
Contrastive learning
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
Evaluation of the Effect of Systolic Blood Pressure and Pulse Pressure on Cognitive Function: The Women's Health and Aging Study II.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏Enhanced skeleton visualization for view invariant human action recognition用于视图不变人体动作识别的增强骨架可视化PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏分享收藏Global spatio-temporal synergistic topology learning for skeleton-based action recognition基于骨架的动作识别的全局时空协同拓扑学习PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
分享收藏A Benchmark Dataset and Comparison Study for Multi-modal Human Action Analytics多模态人类行为分析的基准数据集和比较研究
PREAI
分享收藏 没有更多内容
没有更多内容