.png){kind=icon}
返回
Efficient Two-Stream Network for Online Video Action Segmentation
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2022.3201208.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Temporal action segmentation is a task of predicting frame-level classes of untrimmed long-termed videos. It can be widely used in various applications including customer analysis, data collection, video indexing, and surveillance. Learning good representations from videos such as motion and spatial representation is a critical factor for model performance. Furthermore, the used model should make reliable decisions with low latency for practical use in a resource-constrained environment. However, many works in action segmentation consider just an accuracy using the pre-calculated representations from heavy 3-dimensional(3D) networks. In this paper, we propose a two-stream action segmentation pipeline that can learn motion and spatial information efficiently and operate online. While the temporal stream combines frame-grouping and TSM for capturing short-term dynamics and long-term temporal information at the same time, the spatial stream captures information on color and appearance complementary to representations from the temporal stream. In addition, the results of both the streams are combined by a cross-attention module to provide a desired classification result for the task. Since it can be operated without heavy 3D convolutional neural networks (CNNs), it takes much less memory and computation than conventional 3D-CNN-based methods. Our proposed network using a non-overlapping sliding window achieved segmentation performance on two action segmentation datasets comparable to many recent works that require full temporal resolution and pre-calculated features of 3D CNNs.
Keyword:
Three-dimensional displays
Optical imaging
Motion segmentation
Image color analysis
Streaming media
Solid modeling
Predictive models
Video action segmentation
two-stream network
frame-grouping
temporal shift module
sliding window approach
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Damage Detection and Level Classification of Roof Damage After Typhoon Faxai Based on Aerial Photo and Deep Learning基于航拍照片和深度学习的Faxai台风后顶板损伤检测与等级分类.png)
OAAI
IF0
OAAI分享收藏
查看更多内容