Instance Motion Tendency Learning for Video Panoptic Segmentation

Le Wang

doi:10.1109/TIP.2022.3226414

Instance Motion Tendency Learning for Video Panoptic Segmentation

2023-01-01

3

PRE

AI

DOI：10.1109/TIP.2022.3226414

原文链接

原文求助

摘要

En 中文

Video panoptic segmentation is an important but challenging task in computer vision. It not only performs panoptic segmentation of each frame, but also associates the same instance across adjacent frames. Due to the lack of temporal coherence modeling, most existing approaches often generate identity switches during instance association, and they cannot handle ambiguous segmentation boundaries caused by motion blur. To address these difficult issues, we introduce a simple yet effective Instance Motion Tendency Network (IMTNet) for video panoptic segmentation. It learns a global motion tendency map for instance association, and a hierarchical classifier for motion boundary refinement. Specifically, a Global Motion Tendency Module (GMTM) is designed to learn robust motion features from optical flows, which can directly associate each instance in the previous frame to the corresponding instance in the current frame. In addition, we propose a Motion Boundary Refinement Module (MBRM) to learn a hierarchical classifier to handle the boundary pixels of moving targets, which can effectively revise the inaccurate segmentation predictions. Experimental results on both Cityscapes and Cityscapes-VPS datasets show that our IMTNet outperforms most state-of-the-art approaches.

Keyword:

Image segmentation

Motion segmentation

Task analysis

Tracking

Optical flow

Transformers

Target tracking

Video panoptic segmentation

motion tendency

boundary refinement

deep neural network

Instance Motion Tendency Learning for Video Panoptic Segmentation

摘要

期刊

机构