Efficient Long-Short Temporal Attention network for unsupervised Video Object Segmentation

Ping Li

doi:10.1016/j.patcog.2023.110078

Efficient Long-Short Temporal Attention network for unsupervised Video Object Segmentation

2024-02-01

8

PRE

AI

DOI：10.1016/j.patcog.2023.110078

Original

Original request for help

Save

Abstract

En 中文

Unsupervised Video Object Segmentation (VOS) aims at identifying the contours of primary foreground objects in videos without any prior knowledge. However, previous methods do not fully use spatial-temporal context and fail to tackle this challenging task in real-time. This motivates us to develop an efficient Long -Short Temporal Attention network (termed LSTA) for unsupervised VOS task from a holistic view. Specifically, LSTA consists of two dominant modules, i.e., Long Temporal Memory and Short Temporal Attention. The former captures the long-term global pixel relations of the past frames and the current frame, which models constantly present objects by encoding appearance pattern. Meanwhile, the latter reveals the short-term local pixel relations of one nearby frame and the current frame, which models moving objects by encoding motion pattern. To speedup the inference, the efficient projection and the locality-based sliding window are adopted to achieve nearly linear time complexity for the two light modules, respectively. Extensive empirical studies on several benchmarks have demonstrated promising performances of the proposed method with high efficiency.

Keywords:

Unsupervised video object segmentation

Long temporal memory

Short temporal attention

Efficient projection

Efficient Long-Short Temporal Attention network for unsupervised Video Object Segmentation

Abstract

Journal

Organization