.png){kind=icon}
.png){kind=icon}
Return
Spatial-spectral patch-based multimodal hyperspectral-X data fusion classification network
.png){kind=icon}
PRE
AI
DOI：10.1016/j.engappai.2025.113008.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
With the continuous advancement of remote sensing (RS) technology, the integration of hyperspectral (HS) and X-modality data, such as multispectral (MS), light detection and ranging (LiDAR), and synthetic aperture radar (SAR), is demonstrated to significantly improve the accuracy of land use/land cover classification, which is of great practical significance for large-scale land resource management, urban planning, and environmental monitoring. However, this process still encounters numerous challenges due to disparities in imaging mechanisms, resolutions, and content. Currently, traditional spatial patch-based methods are constrained by the challenge of coupling spatial and spectral data when processing HS images, which hinders the effective extraction of modality-specific information. Moreover, existing multimodal fusion methods lack efficient mechanisms to handle both homogeneous and heterogeneous RS data, thereby limiting their adaptability to complex and diverse feature scenes. To address these issues, a network named spatial–spectral patch-based multimodal hyperspectral-X data fusion classification network (S2PNet) is proposed for multimodal data classification. A novel spectral patch construction (SPC) method is designed to isolate modality-specific information from HS data while completely removing spatial information to capture spectral features more precisely. Additionally, the multi-scale differential convolution (MSDC) is designed to efficiently extract the modality-shared information of HS and X-modality for the characteristics of multimodal RS data. To tackle the common issue of sample imbalance in RS datasets, the equiangular tight frame (ETF) mechanism is introduced into the transformer architecture, which improves the robustness and adaptability of the classification task at the overall framework design level. The experimental results on benchmark datasets containing combinations of HS-MS/LiDAR/SAR demonstrate that S2PNet exhibits significant superiority and advancement in land cover classification tasks, highlighting both its methodological innovation and its potential for real-world RS applications. Compared with state-of-the-art methods, the proposed S2PNet achieves improvements in overall accuracy of 2.51%, 0.02%, 0.18%, and 0.07% on the MUUFL, Trento, Houston2013, and Augsburg datasets, respectively.
Journal
IF:
8
Papers:
5.3K
Citations:
3.5W
Organization
View more organizations