.png){kind=icon}
返回
Cascade Semantic Fusion for Image Captioning
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2019.2917979.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recent advances in deep visual attention methods accelerate greatly the research of image captioning. However, how to leverage hand-crafted features or deep features for the encoder of image captioning is not fully explored, due to the difficulty in finding a kind of all-purpose features to entail a set of visual semantics. In this paper, we introduce a cascade semantic fusion architecture (CSF) to mine the representative features to encode image content through attention mechanism without bells and whistles. Specifically, the CSF benefits from three types of visual attention semantics, including object-level, image-level, and spatial attention features, in a novel three-stage cascade manner. In the first stage, object-level attention features are extracted to capture the detailed contents of the objects based on the pretrained detector. Then, the middle stage devises a fusion module to merge object-level attention features with spatial features, thereby inducing image-level attention features to enrich the context information around the objects. In the last stage, spatial attention features are learned to unveil the salient region representation as a complement to two previously learned attention features. In a nutshell, we integrate attention mechanism with three types of features to organize context knowledge about images from different aspects. The empirical analysis shows that the CSF can assist image captioning model in selecting the object regions of interest. The experiments of image captioning on MSCOCO dataset show the efficacy of our semantic fusion architecture in depicting image content.
Keyword:
Attention mechanism
feature fusion
image captioning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏分享收藏
查看更多内容