.png){kind=icon}
返回
Position-guided transformer for image captioning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.imavis.2022.104575.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Transformer-based frameworks have shown superiorities in image captioning. However, such frameworks are strenuous to consider geometric interrelations among visual contents in an image, as well as fail to prevent changes in the distribution of each layer's input in self-attention. In this work, we first propose a Bi-Positional At-tention (BPA) module, which incorporates absolute and relative position encoding to precisely explore internal relations between objects and their geometric information in an image. Additionally, we use a Group Normaliza-tion (GN) method inside BPA to relieve shifts of the distribution and better exploit the channel dependence of visual features. To validate our proposals, we apply BPA and GN into the original Transformer to constitute our Position-Guided Transformer (PGT) network, which learns a more comprehensive positional representations to augment spatial interactions among objects for image captioning. We conduct extensive experiments to verify the effectiveness of our model. Compared with non-pretraining state-of-the-art methods, experimental results on the MSCOCO benchmark dataset demonstrate that our PGT achieves competitive performance, reaching 134.2% CIDEr score on the Karpathy split with a single model, and 136.2% CIDEr score on the official testing server with an ensemble configuration.(c) 2022 Elsevier B.V. All rights reserved.
Keyword:
Image captioning
Bi-positional attention
Position encoding
Group normalization
Transformer
Self-attention
期刊
IF:
4.2
论文数:
4.1K
被引数:
6.7K
机构
引用论文
.png){kind=icon}
分享收藏分享收藏Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source以五氧化二磷为掺杂源合成n型半导体金刚石膜
PREAI
分享收藏
查看更多内容