Learning From Self-Supervised Features for Hashing-Based Remote Sensing Image Retrieval

doi:10.1109/LGRS.2023.3263498

2023-01-01

PRE

DOI：10.1109/LGRS.2023.3263498

原文链接

原文求助

摘要

En 中文

Image retrieval (IR) for practical remote sensing (RS) should have high accuracy, storage, and calculation efficiency, while not relying on big annotations. However, current supervised and unsupervised RSIR methods do not yet fully meet these requirements. To this end, we propose a novel hashing-based IR approach via learning hash codes from open and representative self-supervised features. Specifically, we constructed a model out of a self-supervised pretrained backbone and a small multilayer perceptron (MLP)-based hashing learning neural network. Features from the frozen backbones were used to reconstruct a similarity matrix to guide the hash network learning. This way, the semantic structure can be preserved. To enhance the proposed approach, we propose the exploitation of global high-level semantic information within the similarity reconstruction process by introducing a small set of labeled datasets. Extensive comparative experiments on two commonly used RS image datasets demonstrate the outperformance of our proposed approach and its good balance between the retrieval accuracy and utilized annotations. In these two datasets, the labeled data required by our method accounts for less than 3% of that required by traditional methods, but our obtained mean average precision (mAP) can reach over 90%, which is close to that of current advanced supervised methods. In addition, we analyzed the specific effect of our design and the associated hyperparameters.

Keyword:

Feature extraction

Training

Semantics

Codes

Neural networks

Image reconstruction

Image retrieval

Deep hashing

few-shot

remote sensing image retrieval (RSIR)

self-supervised features