Enhanced spatial-temporal dynamics in pose forecasting through multi-graph convolution networks

Hongwei Ren

doi:10.1007/s13042-024-02254-9

2024-06-25

PRE

DOI：10.1007/s13042-024-02254-9

原文链接

原文求助

摘要

En 中文

Recently, there has been a growing interest in predicting human motion, which involves forecasting future body poses based on observed pose sequences. This task is complex due to modeling spatial and temporal relationships. Autoregressive models, including recurrent neural networks (RNNs) and their variants, as well as transformer networks, are commonly used for addressing this challenge. However, autoregressive models have several serious drawbacks, such as vanishing or exploding gradients. Other researchers have attempted to solve the communication problem in the spatial dimension by integrating graph convolutional networks (GCNs) and long short-term memory (LSTM) or convolutional neural network (CNN) models. These approaches process temporal and spatial information separately and fuse them to extract features, whereas this sequential processing hampers the model's ability to capture spatiotemporal information and perform feature extraction simultaneously. To address this in human pose forecasting, we propose a novel approach called the multi-graph convolution network (MGCN). By introducing an augmented graph for pose sequences, our model captures spatial and temporal information in one step only using GCN. Multiple frames provide multiple parts, which are joined together in a unified graph instance. Furthermore, our model investigates the impact of natural structure and sequence-aware attention. In the experimental evaluation of the large-scale benchmark datasets (Human3.6M, AMSS, and 3DPW), MGCN outperforms the state-of-the-art methods in human pose prediction.

Keyword:

Graph convolutional network

Pose prediction

Attention mechanism