.png){kind=icon}
Return
Complementary adversarial mechanisms for weakly-supervised temporal action localization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109426.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Weakly supervised Temporal Action Localization (WTAL) aims to locate the start and end boundaries of action instances and recognize their corresponding categories. Classical methods mainly rely on random erasure mechanisms, attention mechanisms, or imposing loss constraints. Despite their great progress, there are still two challenges of incomplete positioning and context confusion. Therefore, we propose a framework with complementary adversarial mechanisms to address these issues. In the adversarial learning stage, for an input snippet, we roughly determine its proper duration, by matching it with the specified multi-scaled anchors based on CAS score loss; then, it undergoes a frame-level iterative regression to precisely figure out its boundary, which can reject none closely related frames merged in, and ensures no overlapping between different action proposals. Subsequently, the GCN module explicitly enhances the feature representation for this fine localized snippet, aiming to strengthen the exclusiveness between different action snippets. Afterward, our complementary learning module calculates the similarity between the original input video V g and the video V r reconstructed with the above localization refined snippets, aiming to ensure no closely relevant frames missing, this checking mechanism works as feedback to guide the regression module for more accurate localization regression. Finally, each refined snippet undergoes multi-instance learning to obtain its classification score, and the top-k strategy is used to aggregate temporally adjacent snippets based on their content similarity, which can avoid fragmentation of an action proposal. This method is tested on datasets of THUMOS14 and ActivityNet1.2, and their average accuracy is 64.68% and 32.94% respectively, its comparisons with other articles prove its effectiveness.(c) 2023 Elsevier Ltd. All rights reserved.
Keywords:
Temporal action localization
Boundary regression
Complementary adversarial mechanism
Weakly supervision learning
Action recognition
Journal
IF:
7.6
Papers:
1.3W
Citations:
4.5W
Organization
No organization information available
Cited Papers
Weakly-supervised action localization via embedding-modeling iterative optimization.png)
PATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSaveLearning principal orientations and residual descriptor for action recognitionPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSaveShareSaveShareSave
ShareSaveWeakly-supervised temporal attention 3D network for human action recognitionPATTERN RECOGNITION
PATTERN RECOGNITIONIF7.6
PREAI
ShareSave
View more