.png){kind=icon}
返回
Gradient Decoupled Learning With Unimodal Regularization for Multimodal Remote Sensing Classification
.png){kind=icon}
PRE
AI
DOI：10.1109/TGRS.2024.3478393.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The joint use of multisource remote-sensing data for Earth observation has drawn much attention due to its robust performance. Although many methods have been proposed to fuse multimodal data, they tend to improve the interaction of different modality data while ignoring the optimization of each modality. Existing studies show that high-performance modalities will suppress the learning of weak ones, leading to under-optimized multimodal learning. To this end, we propose a general framework called gradient decoupled network (GDNet) to assist the multimodal remote sensing (RS) classification. GDNet guides each modality encoder in the multimodal model to learn probabilistic representations instead of deterministic ones. This helps decouple their gradient, reducing their influence on each other and encouraging them to learn the modality-specific information. Then, we further introduce the unimodal regularization for each modality encoder to align their logit output with the multimodal one and label distribution simultaneously. This helps introduce independent gradient paths for each morality encoder to accelerate their optimization when preserving the modality-share information. Finally, extensive experiments conducted on three benchmark datasets demonstrate that the proposed GDNet can effectively address the under-optimized problem in multimodal RS image classification. Code is available at https://github.com/shicaiwei123/TGRS-GDNet.
Keyword:
Feature extraction
Laser radar
Remote sensing
Optimization
Fuses
Transformers
Training
Probabilistic logic
Image classification
Convolutional neural networks
Classification
decoupling learning
deep learning
multimodal
remote sensing (RS)
期刊
IF:
8.6
论文数:
2.2W
被引数:
10.7W
机构
引用论文
Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏Hyperspectral and LiDAR Data Classification Using Joint CNNs and Morphological Feature Learning联合cnn和形态学特征学习的高光谱与LiDAR数据分类
PREAI
分享收藏Coupled adversarial learning for fusion classification of hyperspectral and LiDAR data耦合对抗学习的高光谱与激光雷达数据融合分类.png)
INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
分享收藏Deep EncoderDecoder Networks for Classification of Hyperspectral and LiDAR Data用于高光谱和激光雷达数据分类的深度编码解码器网络
PREAI
分享收藏
查看更多内容