Progressive Frame-Proposal Mining for Weakly Supervised Video Object Detection

Mingfei Han

doi:10.1109/TIP.2024.3364536

Progressive Frame-Proposal Mining for Weakly Supervised Video Object Detection

2024-01-01

2

PRE

AI

DOI：10.1109/TIP.2024.3364536

Original

Original request for help

Save

Abstract

En 中文

In this paper, we focus on the weakly supervised video object detection problem, where each training video is only tagged with object labels, without any bounding box annotations of objects. To effectively train object detectors from such weakly-annotated videos, we propose a Progressive Frame-Proposal Mining (PFPM) framework by exploiting discriminative proposals in a coarse-to-fine manner. First, we design a flexible Multi-Level Selection (MLS) scheme, with explicit guidance of video tags. By selecting object-relevant frames and mining important proposals from these frames, the proposed MLS can effectively reduce frame redundancy as well as improve proposal effectiveness to boost weakly-supervised detectors. Moreover, we develop a novel Holistic-View Refinement (HVR) scheme, which can globally evaluate importance of proposals among frames, and thus correctly refine pseudo ground truth boxes for training video detectors in a self-supervised manner. Finally, we evaluate the proposed PFPM on a large-scale benchmark for video object detection, on ImageNet VID, under the setting of weak annotations. The experimental results demonstrate that our PFPM significantly outperforms the state-of-the-art weakly-supervised detectors.

Keywords:

Proposals

Object detection

Detectors

Annotations

Task analysis

Training

Benchmark testing

Video object detection

weakly supervised learning

holistic-view refinement

Progressive Frame-Proposal Mining for Weakly Supervised Video Object Detection

Abstract

Journal

Organization