.png){kind=icon}
返回
Multi-granularity semantic relational mapping for image caption
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2024.125847.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In terms of constructing object-relationship descriptions in images, existing image captioning methods incorporate regional semantic features into visual features to enhance the visual representation. However, they neglect the construction of grid semantic features, resulting in a lack of accurate detailed relationships in the generated results. We propose a M ulti-granularity S emantic R elational M apping(MSRM) framework that dynamically extracts image semantic cue features in place of traditional region labeling in order to get rid of the semantic capability limitation of fixed classification labels and construct grid semantic features. MSRM use the Internal Semantic Mapping mechanism to refine semantic features by filtering out irrelevant features and mapping them onto region and grid features. Simultaneously, the Semantic Mapping mechanism is used to integrate the composite features derived from regions and grids, thereby addressing the problem of describing semantic relationships among objects across different granularities. Experiments on the MSCOCO and Flickr30k datasets show that the proposed MSRM significantly outperforms the state-of-the-art baselines by more than 4% in 7 different metrics including BLEUs, Meteor, Rouge and CIDEr.
Keyword:
Image caption
Multi-granularity
Dynamical semantic cue
Cross-attention
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
引用论文
分享收藏
Sponge-dwelling snapping shrimps (Alpheidae: Synalpheus) of Barbados, West Indies, with a description of a new eusocial species.png)
Zootaxa
ZootaxaIF0
PREAI
分享收藏Phase Equilibria of Semiclathrate Hydrates of CO2, N2, CH4, or H2+ Tetra-n-butylammonium Bromide Aqueous SolutionPhase Equilibria of Semiclathrate Hydrates of CO2, N2, CH4,或H2 四-n-丁基溴化铵水溶液
PREAI
分享收藏
查看更多内容