.png){kind=icon}
返回
A Multi-Stream Sequence Learning Framework for Human Interaction Recognition
.png){kind=icon}
PRE
AI
DOI：10.1109/THMS.2021.3138708.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Human interaction recognition (HIR) is challenging due to multiple humans' involvement and their mutual interaction in a single frame, generated from their movements. Mainstream literature is based on three-dimensional (3-D) convolutional neural networks (CNNs), processing only visual frames, where human joints data play a vital role in accurate interaction recognition. Therefore, this article proposes a multistream network for HIR that intelligently learns from skeletons' key points and spatiotemporal visual representations. The first stream localises the joints of the human body using a pose estimation model and transmits them to a 1-D CNN and bidirectional long short-term memory to efficiently extract the features of the dynamic movements of each human skeleton. The second stream feeds the series of visual frames to a 3-D convolutional neural network to extract the discriminative spatiotemporal features. Finally, the outputs of both streams are integrated via fully connected layers that precisely classify the ongoing interactions between humans. To validate the performance of the proposed network, we conducted a comprehensive set of experiments on two benchmark datasets, UT-interaction and TV human interaction, and found 1.15% and 10.0% improvement in the accuracy.
Keyword:
Feature extraction
Skeleton
Pose estimation
Computer architecture
Computational modeling
Logic gates
Optical imaging
Bidirectional LSTM (BD-LSTM)
three-dimensional (3-D) convolutional neural network (CNN)
1-D CNN
human interaction recognition (HIR)
human pose estimation
skeleton joint key points
multistream network
期刊
IF:
4.4
论文数:
1.1K
被引数:
3.5K
机构
引用论文
分享收藏Adding Hydrogen to the Isotopic Inventory—Combining δ13C, δ15N and δ2H Stable Isotope Analysis for Palaeodietary Purposes on Archaeological Bone.png)
Archaeometry
OAAI
ArchaeometryIF0
OAAI分享收藏
查看更多内容