.png){kind=icon}
.png){kind=icon}
Return
Embedding Generalized Semantic Knowledge Into Few-Shot Remote Sensing Segmentation
.png){kind=icon}
OA
AI
DOI：10.1109/TGRS.2024.3519772.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Few-shot segmentation (FSS) for remote sensing (RS) imagery leverages supporting information from limited annotated samples to achieve query segmentation of novel classes. Previous efforts are dedicated to mining segmentation-guiding visual cues from a constrained set of support samples. However, they still struggle to address the pronounced intra-class differences in RS images, as sparse visual cues make it challenging to establish robust class-specific representations. In this article, we propose a holistic semantic embedding (HSE) approach that effectively harnesses general semantic knowledge, i.e., class description (CD) embeddings. Instead of the naive combination of CD embeddings and visual features for segmentation decoding, we investigate embedding the general semantic knowledge during the feature extraction stage. Specifically, in HSE, a spatial dense interaction (SDI) module allows the interaction of visual support features with CD embeddings along the spatial dimension via self-attention. Furthermore, a global content modulation (GCM) module efficiently augments the global information of the target category in both support and query features, thanks to the transformative fusion of visual features and CD embeddings. These two components holistically synergize CD embeddings and visual cues, constructing a robust class-specific representation. Through extensive experiments on the standard FSS benchmark, the proposed HSE approach demonstrates superior performance compared to peer work, setting a new state-of-the-art.
Keywords:
Semantics
Visualization
Remote sensing
Feature extraction
Prototypes
Training
Modulation
Decoding
Correlation
Semantic segmentation
Class description (CD) embeddings
few-shot segmentation (FSS)
remote sensing (RS)
semantic embedding
Journal
IF:
8.6
Papers:
2.1W
Citations:
10.7W