.png){kind=icon}
返回
Multimodal Supervised Contrastive Learning in Remote Sensing Downstream Tasks
.png){kind=icon}
PRE
AI
DOI：10.1109/LGRS.2024.3385995.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
To leverage the large amount of unlabeled data available in remote sensing datasets, self-supervised learning (SSL) methods have recently emerged as an ubiquitous tool to pretrain robust image encoder models from unlabeled images. However, when used in a downstream setting, these models often need to be fine-tuned for a specific task after their pretraining. This fine-tuning still requires labeling information to train a classifier on top of the encoder while also updating the encoder weights. In this letter, we investigate the specific task of multimodal scene classification where a sample is composed of multiple views from multiple heterogeneous satellite sensors. We propose a method to improve the categorical cross-entropy fine-tuning process which is often used to specify the model for this downstream task. Our approach, based on the supervised contrastive (SupCon) learning, uses the label information available to train an image encoder in a contrastive manner from multiple modalities while also training the task-specific classifier online. Such a multimodal SupCon loss helps better align representations from samples coming from multiple sensors but having the same class labels, thus improving the performance of the fine-tuning process. Our experiments on two public datasets including DFC2020 and Meter-ML with Sentinel-1/Sentinel-2 images show a significant gain over the baseline multimodal cross-entropy loss. All codes and datasets will be made publicly available for reproducibility upon acceptance.
Keyword:
Task analysis
Self-supervised learning
Remote sensing
Training
Sensors
Optical sensors
Image sensors
Deep learning
multimodality
remote sensing
scene classification
supervised contrastive (SupCon) learning
期刊
IF:
16.4
论文数:
1.0W
被引数:
5.1K
机构
引用论文
.png){kind=icon}
Supervised Contrastive Learning Based on Fusion of Global and Local Features for Remote Sensing Image Retrieval基于全局和局部特征融合的监督对比学习的遥感图像检索
PREAI
分享收藏分享收藏分享收藏The potential of immobilized bacterial α-amylase on coconut coir, a smart carrier for biocatalysts
PREAI
分享收藏Change Detection Based on Supervised Contrastive Learning for High-Resolution Remote Sensing Imagery基于监督对比学习的高分辨率遥感影像变化检测
PREAI
分享收藏
查看更多内容