.png){kind=icon}
返回
In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
.png){kind=icon}
OA
AI
DOI：10.1109/LGRS.2024.3352926.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We investigate the utility of in-domain self-supervised pretraining of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pretraining models before fine-tuning them on a given downstream task. A common approach in practice to SSL pretraining is utilizing standard pretraining datasets, such as ImageNet. While relevant, such a general approach can have a suboptimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this letter, we analyze the effectiveness of SSL pretraining by employing the image bidirectional encoder representations from transformers (BERT) pretraining with online tokenizer (iBOT) framework coupled with Vision transformers (ViTs) trained on million aerial image dataset (Million-AID), a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pretraining strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pretraining consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.
Keyword:
Task analysis
Remote sensing
Data models
Transformers
Training
Self-supervised learning
Scene classification
Deep learning
land use and land cover classification
remote sensing
self-supervised learning (SSL)
期刊
IF:
16.4
论文数:
1.0W
被引数:
5.1K
机构
引用论文
Oxidative stress in liver of grass carp Ctenopharyngodon idella naturally infected with Saprolegnia parasitica and its influence on disease pathogenesis天然感染水蚤的草鱼肝脏氧化应激及其对疾病发病机制的影响
PREAI
分享收藏Evaluation of the Corrosion Protection of Two Underwater Coating Systems in a Simulated Marine Environment在模拟海洋环境中评估两种水下涂层系统的腐蚀防护.png)
JOM
JOMIF0
PREAI
分享收藏Self-Supervised Intra-Modal and Cross-Modal Contrastive Learning for Point Cloud Understanding用于点云理解的自监督模态内和跨模态对比学习
PREAI
分享收藏 没有更多内容
没有更多内容