Temporal consistent portrait video segmentation

doi:10.1016/j.patcog.2021.108143

Temporal consistent portrait video segmentation

2021-12-01

5

PRE

AI

DOI：10.1016/j.patcog.2021.108143

原文链接

原文求助

摘要

En 中文

We explore a new video segmentation task, named portrait video segmentation (PVS), which aims to automatically segment the dominant person throughout a given portrait video. To achieve accurate and temporal-coherent segmentation results, a feature reconstruction based PVS method is developed under the meta-learning framework. Due to the dramatic pose variation and severe occlusion in portrait videos, feature reconstruction using existing optical flow models usually suffers from severe ghosting effects in reconstructed features. We mitigate this issue by presenting a soft correspondence network (SCN), which learns to facilitate feature reconstruction in an unsupervised fashion by softly assigning each pixel displacement probabilities between portrait frames. Based on the proposed SCN, a novel portrait segmentation network (PSN) is further designed, which explores the reconstructed features through feature aggregation blocks (FABs), yielding more reliable segmentation results. To capture temporal and target-specific cues, the parameters of FABs are determined by a meta-updater network which is trained offline in the meta-level. In addition, we introduce a new PVS dataset with high-quality segmentation annotations. Experimental results clearly demonstrate the effectiveness of the proposed PVS method. (c) 2021 Elsevier Ltd. All rights reserved.

Keyword:

Portrait video segmentation

Meta-learning

Feature reconstruction

Feature aggregation block

Temporal consistent portrait video segmentation

摘要

期刊

机构

引用论文