A Generative Pretrained Transformer for Semi-Supervised Hyperspectral Image Change Detection

Yanheng Wang

doi:10.1109/LGRS.2025.3546656

A Generative Pretrained Transformer for Semi-Supervised Hyperspectral Image Change Detection

2025-01-01

0

PRE

AI

DOI：10.1109/LGRS.2025.3546656

Original

Original request for help

Save

Abstract

En 中文

Hyperspectral image change detection (HSIs-CD) often faces the challenge of limited sample sizes, and labeling data is both time-consuming and labor-intensive. Foundation models leverage extensive unlabeled data for self-supervised generative pre-training, allowing the model to learn rich data representations. However, few models have been specifically designed for HSIs, and existing methods often rely on pre-training datasets that are limited to data from a small number of satellite sensors. This limitation affects generalization, especially when there are significant differences between data from different sensors. Moreover, the difference map (DMP) of bi-temporal HSIs is often used as input to the networks. While the DMP-based approach reduces FLOPs, it may lead to information loss compared to dual-branch networks. In this letter, we propose a mini-patch-based generative pre-trained spectral-spatial transformer (GPSST) for semi-supervised HSIs-CD. We begin by collecting public HSIs datasets and dividing them into thousands of patches. Each patch is then split into spectral-spatial tokens, with a portion of these tokens masked and used as input for the GPSST. We then design a spectral-spatial masked autoencoder (MAE) as the backbone of GPSST for self-supervised generative learning. Finally, we fine-tune the GPSST encoder using a small number of labeled patches and design a principal component analysis (PCA) branch to compensate for the information loss caused by the DMP. Our experiments demonstrate that GPSST outperforms existing methods, achieving superior accuracy in HSIs-CD.

Keywords:

Feature extraction

Transformers

Data models

Principal component analysis

Decoding

Hyperspectral imaging

Data mining

Computational modeling

Training

Sensors

Change detection (CD)

generative pre-train transformer

hyperspectral images (HSIs)

semi-supervised

A Generative Pretrained Transformer for Semi-Supervised Hyperspectral Image Change Detection

Abstract

Journal

Organization