.png){kind=icon}
返回
Object semantic analysis for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-023-14596-7.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Although existing image captioning models can produce sentences through attention mechanisms and recurrent neural networks, it is difficult to generate multiple sentences to describe different important objects. Most image captioning models lack description diversity, whereas the diversity models often describe unimportant objects, resulting in low accuracy. In this paper, we propose a novel approach to balancing accuracy and diversity. To achieve this, we designed a novel model which combines saliency information and objects' relative position information to assess the semantic importance of all detected objects. By maintaining the features of important objects and making the network able to describe important objects by operating on the features of unimportant objects, our model can generate sentences with more diversity or accuracy. Experiments demonstrate the characteristics of our model on the MSCOCO and Flickr 30K datasets. In this dataset, our model can provide a set of accurate or diverse descriptions. Compared with the state-of-art models by standard captioning metrics and human evaluation metrics, our model outperforms these works in being able to generate more diverse or accuracy sentences.
Keyword:
Image captioning
Saliency analysis
Relation analysis
Diverse image captioning
期刊
IF:
3
论文数:
2.0W
被引数:
3.2W
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏Fabrication of polymer photonic crystal superprism structures using polydimethylsiloxane soft molds利用聚二甲基硅氧烷软模具制备聚合物光子晶体超棱镜结构
PREAI
分享收藏Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations视觉基因组: 使用众包密集图像注释连接语言和视觉
OAAI
OAAI分享收藏分享收藏
查看更多内容