Self Supervised Progressive Network for High Performance Video Object Segmentation

doi:10.1109/TNNLS.2022.3219936

Self Supervised Progressive Network for High Performance Video Object Segmentation

2024-06-01

4

PRE

AI

DOI：10.1109/TNNLS.2022.3219936

原文链接

原文求助

摘要

En 中文

Recently, self-supervised video object segmentation (VOS) has attracted much interest. However, most proxy tasks are proposed to train only a single backbone, which relies on a point-to-point correspondence strategy to propagate masks through a video sequence. Due to its simple pipeline, the performance of the single backbone paradigm is still unsatisfactory. Instead of following the previous literature, we propose our self-supervised progressive network (SSPNet) which consists of a memory retrieval module (MRM) and collaborative refinement module (CRM). The MRM can perform point-to-point correspondence and produce a propagated coarse mask for a query frame through self-supervised pixel-level and frame-level similarity learning. The CRM, which is trained via cycle consistency region tracking, aggregates the reference & query information and learns the collaborative relationship among them implicitly to refine the coarse mask. Furthermore, to learn semantic knowledge from unlabeled data, we also design two novel mask-generation strategies to provide the training data with meaningful semantic information for the CRM. Extensive experiments conducted on DAVIS-17, YouTube-VOS and SegTrack v2 demonstrate that our method surpasses the state-of-the-art self-supervised methods and narrows the gap with the fully supervised methods.

Keyword:

Task analysis

Customer relationship management

Semantics

Object segmentation

Collaboration

Visualization

Decoding

Cycle consistency

self-supervised

similarity learning

video object segmentation (VOS)

Self Supervised Progressive Network for High Performance Video Object Segmentation

摘要

期刊

机构