Human action recognition using a hybrid deep learning heuristic

Samarendra Chandan Bindu Dash

doi:10.1007/s00500-021-06149-7

2021-08-28

PRE

Samarendra Chandan Bindu Dash

Soumya Ranjan Mishra

K. Srujan Raju *

L V Narasimha Prasad

DOI：10.1007/s00500-021-06149-7

原文链接

原文求助

摘要

En 中文

Human action recognition in the surveillance video is currently one of the challenging research topics. Most of the works in this area are based on either building classifiers on sophisticated handcrafted features or designing deep learning-based convolutional neural networks (CNNs), which directly act on raw inputs and extract meaningful information from the video. To capture the motion information between adjacent frames, 3D CNN extracts features in temporal dimension along with spatial dimension. Even though this technique is very effective in human action recognition but limited to very few fixed frames, all human actions are not limited to a fixed number of frames; they may span several frames. If we increase the size of the input window in CNN, handling all trainable parameters in the network will be very complicated. Hence, it is advisable to encode high-level motion features from different sources to the CNN model. This paper proposed a novel framework to extract handcrafted high-level motion features and in-depth features by CNN in parallel to recognize human action. SIFT is used as handcrafted feature to encode high-level motion features from the maximum number of input video frames. The combination of deep and handcrafted features preserves more extended temporal information from entire video frames present in action video with minimal computational power. Finally, we pass the extracted SIFT into the dense layer and concatenate it with a fully connected layer of CNN for classification. We evaluate the proposed combined CNN framework against regular 3D CNN and traditional handcrafted features like optical flow with SVM, SIFT with SVM on UCF, and KTH human action dataset. We achieve better performance in terms of computational cost and processing time in the proposed CNN framework compared to the other three methods.

Keyword:

Human action recognition

3D Convolutional Neural network

Deep Neural Network

Optical flow

SIFT

Motion features extraction