.png){kind=icon}
返回
Improving skeleton-based action recognition with interactive object information
.png){kind=icon}
OA
AI
DOI：10.1007/s13735-024-00351-7.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Human skeleton information is important in skeleton-based action recognition, which provides a simple and efficient way to describe human pose. However, existing skeleton-based methods focus more on the skeleton, ignoring the objects interacting with humans, resulting in poor performance in recognizing actions that involve object interactions. We propose a new action recognition framework introducing object nodes to supplement absent interactive object information. We also propose Spatial Temporal Variable Graph Convolutional Networks (ST-VGCN) to effectively model the Variable Graph (VG) containing object nodes. Specifically, in order to validate the role of interactive object information, by leveraging a simple self-training approach, we establish a new dataset, JXGC 24, and an extended dataset, NTU RGB+D+Object 60, including more than 2 million additional object nodes. At the same time, we designe the Variable Graph construction method to accommodate a variable number of nodes for graph structure. Additionally, we are the first to explore the overfitting issue introduced by incorporating additional object information, and we propose a VG-based data augmentation method to address this issue, called Random Node Attack. Finally, regarding the network structure, we introduce two fusion modules, CAF and WNPool, along with a novel Node Balance Loss, to enhance the comprehensive performance by effectively fusing and balancing skeleton and object node information. Our method surpasses the previous state-of-the-art on multiple skeleton-based action recognition benchmarks. The accuracy of our method on NTU RGB+D 60 cross-subject split is 96.7%, and on cross-view split, it is 99.2%. The project page: https://github.com/moonlight52137/ST-VGCN.
Keyword:
Graph Convolutional Networks
Action Recognition
Human-Object Interaction
Smart Factory
期刊
IF:
2.9
论文数:
278
被引数:
866
机构
引用论文
DeGCN: Deformable Graph Convolutional Networks for Skeleton-Based Action RecognitionDeGCN: 用于基于骨架的动作识别的可变形图卷积网络OAAI
OAAI分享收藏Evaluation of the Effect of Systolic Blood Pressure and Pulse Pressure on Cognitive Function: The Women's Health and Aging Study II.png)
PLoS ONE
OAAI
PLoS ONEIF0
OAAI分享收藏
查看更多内容