Collaborative Local-Global Learning for Temporal Action Proposal

doi:10.1145/3466181

Collaborative Local-Global Learning for Temporal Action Proposal

2021-09-23

2

PRE

AI

DOI：10.1145/3466181

Original request for help

Save

Abstract

En 中文

Temporal action proposal generation is an essential and challenging task in video understanding, which aims to locate the temporal intervals that likely contain the actions of interest. Although great progress has been made, the problem is still far from being well solved. In particular, prevalent methods can handle well only the local dependencies (i.e., short-term dependencies) among adjacent frames but are generally powerless in dealing with the global dependencies (i.e., long-term dependencies) between distant frames. To tackle this issue, we propose CLGNet, a novel Collaborative Local-Global Learning Network for temporal action proposal. The majority of CLGNet is an integration of Temporal Convolution Network and Bidirectional Long Short-Term Memory, in which Temporal Convolution Network is responsible for local dependencies while Bidirectional Long Short-Term Memory takes charge of handling the global dependencies. Furthermore, an attention mechanism called the background suppression module is designed to guide our model to focus more on the actions. Extensive experiments on two benchmark datasets, THUMOS'14 and ActivityNet-1.3, show that the proposed method can outperform state-of-the-art methods, demonstrating the strong capability of modeling the actions with varying temporal durations.

Keywords:

Temporal action proposal generation

untrimmed videos

long short-term memory

attention mechanism

Collaborative Local-Global Learning for Temporal Action Proposal

Abstract

Journal

Organization