Recurrent Graph Neural Networks for Video Instance Segmentation

doi:10.1007/s11263-022-01703-8

Recurrent Graph Neural Networks for Video Instance Segmentation

2022-11-18

6

OA

AI

DOI：10.1007/s11263-022-01703-8

Original

Original request for help

Save

Abstract

En 中文

Video instance segmentation is one of the core problems in computer vision. Formulating a purely learning-based method, which models the generic track management required to solve the video instance segmentation task, is a highly challenging problem. In this work, we propose a novel learning framework where the entire video instance segmentation problem is modeled jointly. To this end, we design a graph neural network that in each frame jointly processes all detections and a memory of previously seen tracks. Past information is considered and processed via a recurrent connection. We demonstrate the effectiveness of the proposed approach in comprehensive experiments. Our approach operates online at over 25 FPS and obtains 16.3 AP on the challenging OVIS benchmark, setting a new state-of-the-art. We further conduct detailed ablative experiments that validate the different aspects of our approach. Code is available at https://github.com/emibr948/RGNNVIS-PlusPlus.

Keywords:

Detection

Tracking

Segmentation

Video

Recurrent Graph Neural Networks for Video Instance Segmentation

Abstract

Journal

Organization