.png){kind=icon}
Return
Fusing grid and adaptive region features for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.imavis.2025.105513.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Image captioning aims to automatically generate grammatically correct and reasonable description sentences for given images. Improving feature optimization and processing is crucial for enhancing performance in this task. A common approach is to leverage the complementary advantages of grid features and region features. However, incorporating region features in most current methods may lead to incorrect guidance during training, along with high acquisition costs and the requirement of pre-caching. These factors impact the effectiveness and practical application of image captioning. To address these limitations, this paper proposes a method called fusing grid and adaptive region features for image captioning (FGAR). FGAR dynamically explores pseudo-region information within a given image based on the extracted grid features. Subsequently, it utilizes a combination of computational layers with varying permissions to fuse features, enabling comprehensive interaction between information from different modalities while preserving the unique characteristics of each modality. The resulting enhanced visual features provide improved support to the decoder for autoregressively generating sentences describing the content of a given image. All processes are integrated within a fully end-to-end framework, facilitating both training and inference processes while achieving satisfactory performance. Extensive experiments validate the effectiveness of the proposed FGAR method.
Keywords:
Image captioning
Grid feature
Adaptive region feature
Transformer
Feature fusion
Journal
IF:
4.2
Papers:
4.1K
Citations:
6.7K
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSaveAutomatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
OAAI
OAAIShareSave
ShareSaveShareSaveShareSave
View more