.png){kind=icon}
返回
Multi-scale interaction transformer for temporal action proposal generation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.imavis.2022.104589.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Temporal action proposal generation is to localize the time intervals with actions in untrimmed videos. Action instances in untrimmed videos have dramatically varied temporal scales which brings about great challenges for temporal action proposal generation. While temporal action proposal generation has achieved tremendous progress over the past years, multi-scale issue in action proposal generation is still an open problem. In this paper, we propose a Multi-scale Interaction Transformer (MSIT) architecture, which adopts a directly set prediction method to work out the temporal action proposal generation task. MSIT constructs multi-scale feature pyramids and incorporates a novel multi-scale mechanism into Transformer framework. A customized top-down interaction structure is designed to perform self-scale attention and cross-scale attention at different levels of the feature pyramids. Through the top-down interaction, the semantic and location information in each feature level is strengthened and therefore the proposal generation performance can be improved. Furthermore, to model the accurate action locations for each frame, we incorporate an actionness prediction structure to constrain the features output from the encoder. The proposed method was tested on two challenging datasets: THUMOS14 and ActivityNet-1.3. Experiments show that our method achieves comparable performance with state-of-the-art methods. Extensive studies and visualizations also demonstrate the strength of our method. (c) 2022 Elsevier B.V. All rights reserved.
Keyword:
Temporal action proposal generation
Transformer
Multi-scale interaction
Feature pyramids
期刊
IF:
4.2
论文数:
4.1K
被引数:
6.7K
机构
引用论文
3D-TDC: A 3D temporal dilation convolution framework for video action recognition.png)
NEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏I2Net: Mining intra-video and inter-video attention for temporal action localizationNEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容