Patch-Based Transformer Network Construction With Adaptive Feature-Interaction for Hyperspectral Image Classification

doi:10.1109/TGRS.2024.3468387

Patch-Based Transformer Network Construction With Adaptive Feature-Interaction for Hyperspectral Image Classification

2024-01-01

2

PRE

AI

DOI：10.1109/TGRS.2024.3468387

Original

Original request for help

Save

Abstract

En 中文

Hyperspectral image classification (HSIC) is widely used in such fields as vegetation classification and fine agriculture. Nowadays, numerous classification models based on Transformer networks are inputting data typically in the form of patches. However, such patch-based input format may weaken the spatial relationships between neighbor pixels, thus limiting the ability to obtain contextual semantic information. Additionally, hyperspectral images (HSIs) generally contain hundreds of bands, which in turn include some redundant information. The original spectral vectors may lead to redundancy of information, thus reducing the classification accuracy remarkably. Therefore, in this article, a patch-based transformer network construction with adaptive feature-interaction (AFi) for HSIC called AFinet is developed for HSIC. As an end-to-end network, AFinet consists of a feature extraction module and an AFi module. For the developed model, the feed-forward neural network (FNN) in the standard Transformer framework suffers from a limited ability in exploiting local contexts. In this article, an expression-enhanced FNN (E2FNN) is introduced, which can incorporate depthwise convolution layers to capture local contextual information, so as to enhance the correlations between neighbor pixels. Moreover, this study designs an AFi module to facilitate the sharing of interaction features among Transformer-extracted features. Subsequently, global spatial feature information can be integrated into each spectral channel. The AFi module also includes a channel attention mechanism to help focus more attention on channels that contain critical information, while paying less attention to channels with little critical information. After the approach proposed in this study was tested on three widely used datasets, the experimental results demonstrate that the AFinet developed in this study can effectively improve classification accuracy.

Keywords:

Transformers

Feature extraction

Data mining

Convolutional neural networks

Hyperspectral imaging

Convolution

Accuracy

Redundancy

Attention mechanisms

Semantics

Adaptive feature-interaction (AFi)

depthwise convolution

expression-enhanced feed-forward neural network (E2FNN)

hyperspectral image classification (HSIC)

Transformer

Patch-Based Transformer Network Construction With Adaptive Feature-Interaction for Hyperspectral Image Classification

Abstract

Journal

Organization