Skeleton-based 3D human pose estimation with low-resolution infrared array sensor using attention based CNN-BiGRU

Jing Chen

doi:10.1007/s13042-023-02015-0

2023-11-18

PRE

DOI：10.1007/s13042-023-02015-0

原文链接

原文求助

摘要

En 中文

Human sensing based on the low-resolution infrared sensor is widely used in hand gestures recognition, activity recognition, intrusion detection, etc. However, the information about humans acquired by the previous human sensing system using the infrared sensor is limited. In this paper, a human pose estimation system is proposed to realize the three-dimensional skeleton information acquisition by low-resolution infrared sensors. It is a difficult task to acquire human pose estimation with more rich human information from low-resolution infrared sensors. The system leverages the 8 x 8 pixels low-resolution infrared array sensor to collect the activity data and the Kinect v2 camera to capture the three-dimensional skeleton of the human body as annotations of the infrared data. The convolutional neural network-bidirectional gated recurrent unit model with attention mechanism (CNN-BiGRU-AM) model is employed for model training to effectively extract the characteristics of the infrared data from spatial and temporal dimensions. The attention mechanism (AM) can improve the ability of the model to capture important local information. The bone joint point data predicted by the model are utilized to draw the three-dimensional skeleton diagram. The k-means clustering algorithm is applied to eliminate the outliers that affect the overall visualization effect in the prediction. The accuracy and completeness of human pose estimation are measured by the euclidean distance between the real coordinates of the bone joint points obtained by Kinect v2 camera and the coordinates predicted by the model. The proportion of the number of predictions with euclidean distance less than a threshold 20 mm is 90.151%, representing the accuracy of human pose estimation. The experimental results show that three-dimensional skeleton information can be acquired accurately by the low-resolution infrared array sensor and the subtle difference within each activity can be observed through the 3D human pose to improve the effect of activity recognition.

Keyword:

Skeleton-based 3D human pose estimation

Low-resolution infrared array sensor

Convolutional neural network (CNN)

Bidirectional gated recurrent unit (BiGRU)

Attention mechanism