Incremental Vision–Language Object Detection via Sparse Frequency Transform

Xiang Song

doi:10.1109/tip.2026.3723720

Incremental Vision–Language Object Detection via Sparse Frequency Transform

2026-08-19

0

PRE

AI

GONG YIHONG (Yihong Gong)

DOI：10.1109/tip.2026.3723720

Original

Original request for help

Save

Abstract

En 中文

Vision-language object detectors (VLODs) pretrained on large image-text corpora exhibit strong zero-shot detection abilities, yet their performance often degrades on specialized downstream tasks that arrive sequentially. Hence, we study Incremental Vision-Language Object Detection (IVLOD), which requires continually adapting a VLOD across tasks with domain and/or class shifts while mitigating catastrophic forgetting of prior tasks and preserving its zero-shot generalization. To address this problem, we propose a novel method, named Sparse Frequency Transform (SFT), that minimizes inter-task interference via a frequency-domain design. Building on a theoretical link between forgetting and Frobenius inner product (FIP), SFT enforces sparsity-independence on spectral supports, yielding zero inter-task FIP by construction and achieving reduced forgetting with low impact on the pretrained model. Spectral supports are selected via an online gradient-magnitude scoring rule with masking of previously used supports, then converted via inverse discrete cosine transform (IDCT) to dense time-domain updates that preserve FIP constraints and improve optimization efficiency. Extensive experiments under both full-shot and few-shot IVLOD settings demonstrate that our SFT is capable of consistently learning new tasks while preserving the zero-shot generalization capabilities of the pretrained model.

Keywords:

Incremental object detection

vision-language model

catastrophic forgetting

discrete cosine transform

Incremental Vision–Language Object Detection via Sparse Frequency Transform

Abstract

Journal

Organization

Cited Papers