.png){kind=icon}
返回
Reference-based model using multimodal gated recurrent units for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-020-09539-5.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Describing images through natural language is a challenging task in the field of computer vision. Image captioning consists of creating image descriptions that can be accomplished via deep learning architectures that use convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, traditional RNNs encounter problems such as exploding and vanishing gradients, and they exhibit poor performance when generating non-descriptive sentences. To solve these issues, we proposed a model based on the encoder-decoder structure using CNNs to extract the image features and multimodal gated recurrent units (GRU) for descriptions. This model implements the part-of-speech (PoS) and likelihood function for weight generation in the GRU. The method performs knowledge transfer during a validation phase that uses the k-nearest neighbors technique (kNN). Experimental results using the Flickr30k and MSCOCO datasets demonstrated that the proposed PoS-based model presents competitive scores in comparison to state-of-the-art models. The system predicts more descriptive captions and closely approximates the expected captions both in the predicted andkNN selected captions.
Keyword:
Gated recurrent units
Caption generation references
Convolutional neural network
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3
论文数:
2.0W
被引数:
3.2W
机构
引用论文
.png){kind=icon}
Free, quasifree, momentarily trapped, and self-trapped exciton states in molecular crystals自由、准自由、瞬间捕获和自捕获激子态在分子晶体中
PREAI
分享收藏分享收藏
查看更多内容