Regularized Masked Auto-Encoder for Semi-Supervised Hyperspectral Image Classification

Liguo Wang

doi:10.1109/TGRS.2024.3509720

Regularized Masked Auto-Encoder for Semi-Supervised Hyperspectral Image Classification

2024-01-01

0

PRE

AI

DOI：10.1109/TGRS.2024.3509720

Original

Original request for help

Save

Abstract

En 中文

As the most prevalent self-supervised representation learning (SSRL) model, the masked auto-encoder (MAE) has been gradually investigated in semi-supervised hyperspectral image classification (SHIC). However, the majority of the current approaches augment MAE merely from the application perspective or by introducing a weak regularization term, and do not comprehensively consider the challenges posed by the high intraclass variances and interclass similarities that often appear in hyperspectral image (HSI) data. In this article, we present a regularized MAE (RMAE) to address the aforementioned problems. Specifically, within the framework of MAE, we introduce a self-designed induced transformer block, using a small number of visible patches to learn the embeddings of patches with larger receptive fields. The learned embeddings are used to reconstruct the corresponding patches and an induced reconstruction loss is calculated. This strategy creates a much harder task for masked image modeling (MIM), and the induced transformer block is lightweight and imposes negligible computational burden overhead the underlying MAE framework. In addition, by rethinking the masking operations, we develop a masked convolutional neural network (MCNN), uncovering the principle of MAE and affirming the efficacy of RMAE. Finally, we present two metrics: the mean intraclass distance, and the mean interclass distance. Based on the metrics we give two criteria to evaluate the performance of an SSRL model, providing a new coordinate for the research in SSRL-based SHIC. Experiments conducted on four publicly accessible datasets show that RMAE outperforms state-of-the-art methods. The source code was powered by Jupyter and released at https://github.com/swiftest/RMAE.

Keywords:

Transformers

Feature extraction

Training

Three-dimensional displays

Convolution

Tuning

Sun

Representation learning

Iterative methods

Hyperspectral imaging

Induced transformer

masked image modeling (MIM)

regularized masked auto-encoder (MAE)

self-supervised representation learning (SSRL)

semi-supervised hyperspectral image classification (SHIC)

Regularized Masked Auto-Encoder for Semi-Supervised Hyperspectral Image Classification

Abstract

Journal

Organization