.png){kind=icon}
返回
Prior Knowledge-Guided Transformer for Remote Sensing Image Captioning
.png){kind=icon}
PRE
AI
DOI：10.1109/TGRS.2023.3328181.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Remote sensing image (RSI) captioning aims to generate meaningful and grammatically accurate sentences for RSIs. However, in comparison to natural image captioning, RSI captioning encounters additional challenges due to the unique characteristics of RSIs. The first challenge arises from the abundance of objects present in these images. As the number of objects increases, it becomes increasingly difficult to determine the main focus of the description. Moreover, the objects in RSIs often share similar appearances, which further complicates the generation of accurate descriptions. To overcome these challenges, we propose a prior knowledge-guided transformer (PKG-Transformer) for RSI captioning. First, scene-level and object-level features are extracted in a multilevel feature extraction (MFE) module. To further refine and enhance the extracted multilevel features, we introduce a feature enhancement (FE) module. This module utilizes a combination of graph neural networks and attention mechanisms to capture the correlation and difference between different objects or scene regions. Moreover, we propose a prior knowledge augmented attention (PKA) mechanism to select the objects that are more relevant to the scene regions by establishing the relationships between them. This attention mechanism is seamlessly integrated into the transformer structure, providing valuable prior knowledge that promotes the caption generation process. Extensive experiments on three RSI captioning datasets verify the superiority of the proposed method. Compared with the baseline methods, the proposed method achieves more impressive performance. The code will be publicly available at https://github.com/One-paper-luck/PKG-Transformer
Keyword:
Feature extraction
Transformers
Remote sensing
Task analysis
Iron
Decoding
Convolutional neural networks
Image captioning
prior knowledge
remote sensing
transformer
期刊
IF:
8.6
论文数:
2.2W
被引数:
10.7W
机构
引用论文
.png){kind=icon}
Evidence of Multiple Crystallization Pathways in Lithium Disilicate: A Metadynamics Investigation二硅酸锂中多种结晶途径的证据: 元动力学研究
PREAI
分享收藏
分享收藏分享收藏Exploring Transformer and Multilabel Classification for Remote Sensing Image Captioning探索用于遥感图像字幕的变压器和多标签分类
OAAI
OAAI分享收藏
查看更多内容