Full-duplex strategy for video object segmentation

Ge-Peng Ji

doi:10.1007/s41095-021-0262-4

Full-duplex strategy for video object segmentation

2023-03-01

5

OA

AI

DOI：10.1007/s41095-021-0262-4

Original

Original request for help

Save

Abstract

En 中文

Previous video object segmentation approaches mainly focus on simplex solutions linking appearance and motion, limiting effective feature collaboration between these two cues. In this work, we study a novel and efficient full-duplex strategy network (FSNet) to address this issue, by considering a better mutual restraint scheme linking motion and appearance allowing exploitation of cross-modal features from the fusion and decoding stage. Specifically, we introduce a relational cross-attention module (RCAM) to achieve bidirectional message propagation across embedding sub-spaces. To improve the model's robustness and update inconsistent features from the spatiotemporal embeddings, we adopt a bidirectional purification module after the RCAM. Extensive experiments on five popular benchmarks show that our FSNet is robust to various challenging scenarios (e.g., motion blur and occlusion), and compares well to leading methods both for video object segmentation and video salient object detection. The project is publicly available at https://github.com/GewelsJI/FSNet.

Keywords:

video object segmentation (VOS)

video salient object detection (V-SOD)

visual attention

Full-duplex strategy for video object segmentation

Abstract

Journal

Organization