.png){kind=icon}
返回
Learning joint relationship attention network for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2022.118474.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image captioning aims at automatically describing the main content of an image with a complete and natural sentence. Existing attention-based methods often focus on visual features individually, while ignoring relationship information between image features that provides important guidance for generating captions. To alleviate this issue, in this work we propose the new Joint Relationship Attention Network (JRAN) that novelly explores the relationships between the features in the image. Technically, JRAN capitalizes on semantic features as s supplementary to the region features, fully learn two types of relationships, the visual relationships between region features and the visual-semantic relationships between region features and semantic features. Then, JRAN further make a dynamic trade-off between them during outputting the relationship representation. Moreover, we devise a new feature fusion based attention, which can adaptively fuse the region features and previously obtained relationship representation when generating different words. Extensive experiments on MSCOCO and Flickr30k, Flickr8k datasets show the superiority of our JRAN method qualitatively and quantitatively compared with several related state-of-the-art methods. More remarkably, JRAN achieves 28.3% and 58.2% on Flickr30k, and 22.7% and 55.3% on Flickr8k for the metrics BLEU4 and CIDEr.
Keyword:
Visual relationship
Feature relationship network
Joint relationship learning
Image captioning
期刊
IF:
7.5
论文数:
3.0W
被引数:
10.2W
机构
引用论文
.png){kind=icon}
A prospective study of surgical margin status in oral squamous cell carcinoma: A preliminary report
PREAI
分享收藏
分享收藏分享收藏
查看更多内容