Controllable Generative Knowledge-Driven Few-Shot Object Detection From Optical Remote Sensing Imagery

doi:10.1109/TGRS.2025.3541937

Controllable Generative Knowledge-Driven Few-Shot Object Detection From Optical Remote Sensing Imagery

2025-01-01

0

PRE

AI

DOI：10.1109/TGRS.2025.3541937

Original

Original request for help

Save

Abstract

En 中文

Few-shot object detection (FSOD) has to learn classification and localization information for unseen object detection under very low-data resource regimes. However, when deficient samples are adopted for model training, it is hard to build powerful location-aware and identification abilities for well coping with agnostic bias from diverse testing scenarios; at the same time, the overfitting phenomenon is easily occurring. Therefore, in this article, a controllable generative knowledge-driven FSOD called CGK-FSOD is proposed for unseen object detection from optical remote sensing imagery. Specifically, to enrich the learnable data space of scarce samples for preventing incomplete agnostic-bias learning, while avoiding the overfitting phenomenon, a visual-textual prompt-based controllable data generation is designed to generate high-quality object detection data based on pretrained foundational models [i.e., the stable diffusion (SD) and contrastive language-image pre-training (CLIP)], which not only can introduce the generalized domain-level knowledge into the remote sensing domain but also sets up an all-round data space to support complete learning of potential agnostic bias. Furthermore, with respect to the denoising generative process of SD, a series of cross-modality generative features in latent representation space are reused for few-shot fine-tuning by the designed cross-modality feature embedding (CMFE), which not only can bring diverse generative abilities into the feature fusion step of the detector but also gracefully sets up feature representation scalability to make the detector better adapt to agnostic bias from diverse testing scenarios of FSOD. Finally, extensive experiments are executed on two public remote sensing datasets (e.g., DIOR and NWPUVHR-10), and the results indicate that the proposed CGK-FSOD is very effective and flexible for FSOD.

Keywords:

Training

Detectors

Object detection

Feature extraction

Remote sensing

Testing

Overfitting

Data models

Few shot learning

Data collection

Contrastive language-image pre-training (CLIP)

controllable generation

few-shot object detection (FSOD)

remote sensing

stable diffusion (SD)

visual-textual prompt

Controllable Generative Knowledge-Driven Few-Shot Object Detection From Optical Remote Sensing Imagery

Abstract

Journal

Organization