Multi-granular spatial-temporal synchronous graph convolutional network for robust action recognition

doi:10.1016/j.eswa.2024.124980

Multi-granular spatial-temporal synchronous graph convolutional network for robust action recognition

2024-12-01

0

PRE

AI

DOI：10.1016/j.eswa.2024.124980

Original

Original request for help

Save

Abstract

En 中文

Graph Convolutional Networks (GCNs) have shown great potential in skeleton-based human action recognition. However, due to the diversity and complexity, modeling human actions as general graphs and capturing discriminative spatial-temporal motion patterns is challenging. Besides, the inevitable interference, especially occlusion, impairs the robustness of existing methods that depend on complete skeletons. To solve these problems, we propose a Multi-Granular Spatial-Temporal Synchronous Graph Convolutional Network (MSSGCN). Firstly, we investigate three partition strategies: attribute, activity, and mixed partition strategy to optimize the weight-sharing mechanism of GCNs, which facilitates the novel Extended Adaptive Graph Convolution (EAGC) module. Secondly, we elaborate on a Multi-sliced Spatial-temporal Graph (MSTG) for multi-granular action modeling. Thirdly, we present a Synchronized Slice Encoder (Syn-STE) to simultaneously embed spatial and temporal action patterns. Then, we design Multi-granular Spatial-temporal Encoders (MultiSTE) with multi-branch Syn-STE to generate multi-granular context. The extensive experiments verified that MSS-GCN is more robust and outperforms benchmarks on NTU-RGB+D, NTU-RGB+D 120, and NW-UCLA datasets.

Keywords:

Action recognition

Graph convolutional networks

Spatial-temporal modeling

Multi-granular analysis

Multi-granular spatial-temporal synchronous graph convolutional network for robust action recognition

Abstract

Journal

Organization