Skeleton-based structured early activity prediction

doi:10.1007/s11042-020-08875-w

Skeleton-based structured early activity prediction

2020-04-24

2

PRE

AI

M

Mohammad Mahdi Arzani

DOI：10.1007/s11042-020-08875-w

Original

Original request for help

Save

Abstract

En 中文

To communicate with people, robots and vision-based interactive systems often need to understand human activities in advance before the activity is performed completely. This early prediction of the activities will help them take proper near future steps to fulfill a realistic interactive session with humans. However, predicting activities in advance is a very challenging task, because some activities are simple while others are complex and comprised of several smaller atomic sub-activities. In this paper, we propose a method capable of early prediction of simple and complex human activities by formulating it as a structured prediction task using probabilistic graphical models (PGM). We use skeletons captured from low-cost depth sensors as high-level descriptions of the human body. Using 3D skeletons, our method will be robust to the environmental factors. Our proposed model is a fully observed PGM coupled with a clustering scheme to remove the dependency of our model to the number-of-middle-states hyperparameter. We test our method on three popular datasets: CAD-60, UT-Kinect, and Florence 3D and obtain accuracies of 97.6% , 100% and 96.11%, respectively. These datasets cover both simple and complex activities. When only half of the clip is observed, we achieve 93.33% and 96.9% accuracy on CAD-60 and UT-Kinect datasets, respectively.

Keywords:

Probabilistic graphical models

Human activity early prediction

Distributed structured prediction

Latent structured support vector machines

Robot vision

3D skeleton

RGB-D

Skeleton-based structured early activity prediction

Abstract

Journal

Organization