Automatic image caption generation using deep learning

doi:10.1007/s11042-023-15555-y

2023-06-01

PRE

DOI：10.1007/s11042-023-15555-y

原文链接

原文求助

摘要

En 中文

Image captioning is an interesting and challenging task with applications in diverse domains such as image retrieval, organizing and locating images of users' interest, etc. It has huge potential for replacing manual caption generation for images and is especially suitable for large-scale image data. Recently, deep neural network based methods have achieved great success in the field of computer vision, machine translation, and language generation. In this paper, we propose an encoder-decoder based model that is capable of generating grammatically correct captions for images. This model makes use of VGG16 Hybrid Places 1365 as an encoder and LSTM as a decoder. To ensure the complete ground truth accuracy, the model is trained on the labeled Flickr8k and MS-COCO Captions datasets., Further, the model is evaluated using all popular standard metrics such as BLEU, METEOR, GLEU, and ROUGE_L. Experimental results indicate that the proposed model obtained a BLEU-1 score of 0.6666, METEOR score of 0.5060, and GLEU score of 0.2469 on the Flickr8k dataset and BLEU-1 score 0.7350, METEOR score of 0.4768 and GLEU score 0.2798 on MS-COCO Caption dataset. Thus, the proposed method achieved a significant performance as compared to the state-of-art approaches. To evaluate the efficacy of the model further, we also show the results of caption generation from live sample images that reinforce the validity of the proposed approach.

Keyword:

Image

Neural network

Caption

CNN (Convolutional Neural Network)

Feature extraction

RNN (Recurrent Neural Network)

LSTM (Long Short-Term Memory)