.png){kind=icon}
返回
NumCap: A Number-controlled Multi-caption Image Captioning Network
.png){kind=icon}
PRE
AI
DOI：10.1145/3576927.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image captioning is a promising task that attracted researchers in the last few years. Existing image captioning models are primarily trained to generate one caption per image. However, an image may contain rich contents, and one caption cannot express its full details. A better solution is to describe an image with multiple captions, with each caption focusing on a specific aspect of the image. In this regard, we introduce a new number-based image captioning model that describes an image with multiple sentences. An image is annotated with multiple ground-truth captions; thus, we assign an external number to each caption to distinguish its order. Given an image-number pair as input, we could achieve different captions for the same image under different numbers. First, a number is attached to the image features to form an image-number vector (INV). Then, this vector and the corresponding caption are embedded using the order-embedding approach. Afterward, the INV's embedding is fed to a language model to generate the caption. To show the efficiency of the numbers incorporation strategy, we conduct extensive experiments using MS-COCO, Flickr30K, and Flickr8K datasets. The proposed model attains 24.1 in METEOR on MS-COCO. The achieved results demonstrate that our method is competitive with a range of state-of-the-art models and validate its ability to produce different descriptions under different given numbers.
Keyword:
Numbers incorporation strategy
encoder-decoder framework
image captioning
order-embedding
期刊
IF:
6
论文数:
2.0K
被引数:
5.4K
机构
引用论文
.png){kind=icon}
Cross-Domain Image Captioning via Cross-Modal Retrieval and Model Adaptation通过跨模态检索和模型自适应实现跨域图像字幕
PREAI
分享收藏分享收藏分享收藏An Ensemble of Generation- and Retrieval-Based Image Captioning With Dual Generator Generative Adversarial Network具有双生成器生成对抗网络的基于生成和检索的图像字幕的集成
PREAI
分享收藏
查看更多内容