STA-CNN: Convolutional Spatial-Temporal Attention Learning for Action Recognition

doi:10.1109/TIP.2020.2984904

STA-CNN: Convolutional Spatial-Temporal Attention Learning for Action Recognition

2020-01-01

62

OA

AI

DOI：10.1109/TIP.2020.2984904

Original

Original request for help

Save

Abstract

En 中文

Convolutional Neural Networks have achieved excellent successes for object recognition in still images. However, the improvement of Convolutional Neural Networks over the traditional methods for recognizing actions in videos is not so significant, because the raw videos usually have much more redundant or irrelevant information than still images. In this paper, we propose a Spatial-Temporal Attentive Convolutional Neural Network (STA-CNN) which selects the discriminative temporal segments and focuses on the informative spatial regions automatically. The STA-CNN model incorporates a Temporal Attention Mechanism and a Spatial Attention Mechanism into a unified convolutional network to recognize actions in videos. The novel Temporal Attention Mechanism automatically mines the discriminative temporal segments from long and noisy videos. The Spatial Attention Mechanism firstly exploits the instantaneous motion information in optical flow features to locate the motion salient regions and it is then trained by an auxiliary classification loss with a Global Average Pooling layer to focus on the discriminative non-motion regions in the video frame. The STA-CNN model achieves the state-of-the-art performance on two of the most challenging datasets, UCF-101 (95.8%) and HMDB-51 (71.5%).

Keywords:

Videos

Feature extraction

Motion segmentation

Computational modeling

Image recognition

Solid modeling

Convolutional neural networks

Temporal attention

spatial attention

convolutional neural network

action recognition

STA-CNN: Convolutional Spatial-Temporal Attention Learning for Action Recognition

Abstract

Journal

Organization