Multi-Attention Generative Adversarial Network for image captioning

Yiwei Wei

doi:10.1016/j.neucom.2019.12.073

Multi-Attention Generative Adversarial Network for image captioning

2020-04-01

24

PRE

AI

DOI：10.1016/j.neucom.2019.12.073

原文链接

原文求助

摘要

En 中文

Recently, it has been shown that generative-adversarial-nets (GANs) can be directly utilized as an extension of traditional reinforcement-learning in image captioning tasks. However, the GANs-based methods generate captions as a function of only local points in the feature map without capturing non-local information. In this paper, a Multi-Attention mechanism is first proposed by utilizing both of the local and non-local evidence for more effective feature representation and reasoning in image captioning. Based on the mechanism, a Multi-Attention Generative Adversarial Image Captioning Network (MAGAN) is also proposed which contains a Multi-Attention generator and a Multi-Attention discriminator. The proposed generator is designed to generate more accurate sentences, while the proposed discriminator is employed to determine whether generated sentences are human described or machine generated. Extensive experiments are conducted to validate the proposed framework on MSCOCO benchmark dataset, and it achieves very competitive results evaluated by the evaluation server of MS COCO captioning challenge. (C) 2019 Published by Elsevier B.V.

Keyword:

Image captioning

Multi-Attention

GANs

Reinforcement learning

Multi-Attention Generative Adversarial Network for image captioning

摘要

AI总结

期刊

机构

引用论文