.png){kind=icon}
.png){kind=icon}
Return
Criss-Cross Attention Enhanced Transformer for Image Captioning
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-031-97352-9_6.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The transformer model offers substantial advantages across various applications, including natural language processing, machine translation, and image captioning. It effectively handles long-range dependencies and allows for parallel computations. However, transformers face limitations in capturing fine-grained dependencies and ensuring efficient information flow between the encoder and decoder. To address these gaps, we optimize the transformer architecture by introducing a criss-cross attention module. This module gathers contextual information in both horizontal and vertical directions. As a result, the model captures intricate dependencies and spatial relationships more effectively, leading to a comprehensive understanding of the input data. In addition, we utilize mesh connections during the decoder stage to leverage the low and high level features. This fully connected approach between the encoder and the decoder facilitates more effective bidirectional information transfer, enhancing the contextual understanding and coherence of the generated outputs. Extensive experiments on the MSCOCO benchmark reveal that our enhanced model generates more accurate and descriptive captions. Our model demonstrates superior performance in metrics such as BLEU, METEOR, and CIDEr.
Keywords:
image captioning
criss-cross attention
transformer
mesh connections
Journal
D
IF:
0
Papers:
25
Citations:
0