.png){kind=icon}
返回
StochasticFormer: Stochastic Modeling for Weakly Supervised Temporal Action Localization
.png){kind=icon}
PRE
AI
DOI：10.1109/TIP.2023.3244411.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Weakly supervised temporal action localization (WS-TAL) aims to identify the time intervals corresponding to actions of interest in untrimmed videos with video-level weak supervision. For most existing WS-TAL methods, two commonly encountered challenges are under-localization and over-localization, which inevitably bring about severe performance deterioration. To address the issues, this paper proposes a transformer-structured stochastic process modeling framework, namely StochasticFormer, to fully investigate finer-grained interactions among the intermediate predictions to achieve further refined localization. StochasticFormer is built on a standard attention-based pipeline to derive preliminary frame/snippet-level predictions. Then, the pseudo localization module generates variable-length pseudo action instances with the corresponding pseudo labels. Using the pseudo action instance - action category pairs as fine-grained pseudo supervision, the stochastic modeler aims to learn the underlying interaction among the intermediate predictions with an encoder-decoder network. The encoder consists of the deterministic and latent path to capture the local and global information, which are subsequently integrated by the decoder to obtain reliable predictions. The framework is optimized with three carefully designed losses, i.e. the video-level classification loss, the frame-level semantic coherence loss, and the ELBO loss. Extensive experiments on two benchmarks, i.e., THUMOS14 and ActivityNet1.2, have shown the efficacy of StochasticFormer compared with the state-of-the-art methods.
Keyword:
Location awareness
Stochastic processes
Feature extraction
Videos
Transformers
Training
Annotations
Temporal action localization
action recognition
stochastic process
期刊
IF:
13.7
论文数:
1.0W
被引数:
8.4W
机构
引用论文
.png){kind=icon}
分享收藏
Defining Molecular Signature of Pro-Immunogenic Radiotherapy Targets in Human Prostate Cancer Cells
OAAI
OAAI分享收藏
查看更多内容