.png){kind=icon}
Return
Remote Sensing Image-Text Retrieval With Implicit-Explicit Relation Reasoning
.png){kind=icon}
PRE
AI
DOI：10.1109/TGRS.2024.3466909.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Remote sensing image-text retrieval (RSITR) has become a research hotspot in recent years for its wide application. Existing methods in this context, based either on local or global feature matching, overlook the sensing variation-leaded visual deviation and geographically nearby image-text mismatching problems of remote sensing (RS) images. This work notes that this would limit the retrieval accuracy for RSITR. To handle this, we present IERR, an implicit-explicit relation reasoning framework that learns relations between local visual-textual tokens and enhances global image-text matching without requiring additional prior supervision. Specifically, masked image modeling (MIM) and masked language modeling (MLM) are used for symmetric mask reasoning consistency alignment. Meanwhile, masked features (i.e., implicit relation) and unmasked features (i.e., explicit relation) are fed into a multimodal interaction encoder to enhance the representations of the textual-visual features. Extensive experimental results on the RSICD and RSITMD datasets demonstrate the superiority of IERR compared with 17 baselines.
Keywords:
masked image modeling (MIM)
Implicit relation reasoning
Implicit relation reasoning
Implicit relation reasoning
Implicit relation reasoning
Implicit relation reasoning
masked language modeling (MLM)
masked language modeling (MLM)
masked language modeling (MLM)
masked image modeling (MIM)
masked language modeling (MLM)
remote sensing image-text retrieval (RSITR)
Journal
IF:
8.6
Papers:
2.2W
Citations:
10.7W
Organization
View more organizations
Cited Papers
ShareSaveCLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
ShareSaveShareSaveShareSave
View more