.png){kind=icon}
返回
A reference-based model using deep learning for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-022-00937-3.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Describing images in natural language is a challenging task for computer vision. Image captioning is the task of creating image descriptions. Deep learning architectures that use convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are beneficial in this task. However, traditional RNNs may cause problems, including exploding gradients, vanishing gradients, and non-descriptive sentences. To solve these problems, we propose a model based on the encoder-decoder structure, using CNNs to extract features from reference images and gated recurrent units (GRUs) to create the descriptions. Our model applies part-of-speech (PoS) analysis and the likelihood function to generate weights in GRU. This method also performs the knowledge transfer during a validation phase using the k-nearest neighbors (kNN) technique. Our experimental results using Flickr30k and MS-COCO datasets indicate that the proposed PoS-based model yields competitive scores compared to those of high-end models. The system predicts more descriptive captions and closely approximates the expected captions for both the predicted and kNN-selected captions.
Keyword:
Gated recurrent units
Caption generation references
Convolutional neural network
Part-of-speech tagging
期刊
IF:
3.1
论文数:
2.8K
被引数:
2.7K
机构
引用论文
.png){kind=icon}
Free, quasifree, momentarily trapped, and self-trapped exciton states in molecular crystals自由、准自由、瞬间捕获和自捕获激子态在分子晶体中
PREAI
分享收藏分享收藏
分享收藏Investigation of Ti mesh-supported anodes for direct borohydride fuel cells直接硼氢化物燃料电池用钛网支撑阳极的研究
PREAI
分享收藏
查看更多内容