Adaptive multi-scale framework for incremental semantic segmentation

Shaojun Qu

doi:10.1007/s10044-025-01564-0

Return

Adaptive multi-scale framework for incremental semantic segmentation

2025-10-25

PRE

Shaojun Qu *

Dong‐Lin Xie

DOI：10.1007/s10044-025-01564-0

Original

Original request for help

Save

Abstract

En 中文

Class-incremental semantic segmentation (CISS) aims to progressively learn new categories or adapt to novel environments for semantic segmentation tasks without requiring full model retraining, while ensuring that segmentation performance on previously learned classes is preserved. Most existing class-incremental semantic segmentation methods mitigate the issues of catastrophic forgetting and background shift through strategies such as pseudo-labeling and knowledge distillation. Although existing methods have achieved certain success, they still have some limitations: (1) they ignore contextual relationships between tasks, resulting in a lack of holistic learning; (2) knowledge from the old model is directly transferred to the new model, even though not all of it is beneficial for the new model. This paper proposes a novel Adaptive Multi-scale Incremental Semantic Segmentation (AMIS) framework that incorporates a Global Attention Block (GAB), designed to capture contextual information across different tasks and address these challenges. In addition, this work employs an Adaptive Multi-scale Distillation (AMD) module to perform multi-scale pooling and fusion on features extracted by the decoder, enabling the model to adaptively focus on informative representations. Moreover, a Background Compensation Strategy (BCS) is applied to enhance the model’s ability to distinguish ambiguous boundaries between background and target classes. Extensive experiments on the Pascal VOC and ADE20K datasets demonstrate that the proposed method effectively mitigates catastrophic forgetting and background shift, outperforming state-of-the-art approaches in most scenarios. The code is available at https://github.com/ZXCV-7/AMIS .

Keywords:

Incremental learning

Incremental semantic segmentation

Adaptive distillation

Multi-scale pooling

Background compensation strategy