.png){kind=icon}
返回
Improving Image Captioning Systems With Postprocessing Strategies
.png){kind=icon}
PRE
AI
DOI：10.1109/TGRS.2023.3281334.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Image captioning (IC) systems are generally based on encoder-decoder architecture where convolutional neural networks (CNNs) are employed to represent an image with discriminative features and recurrent neural networks (RNNs) sequentially generate a sentence description. Even though a lot of effort has been devoted lately to designing reliable IC systems, the task is far from being solved. The generated descriptions can be affected by different errors related to the attributes and the objects present in the scene. Moreover, once an error occurs, it can be propagated in the recurrent layers of the decoder generating non-accurate descriptions. To solve this problem, we propose two postprocessing strategies applied to the generated descriptions to rectify the errors and improve their quality. The proposed postprocessing strategies are based on hidden Markov models (HMMs) and Viterbi algorithm. The proposed postprocessing strategies can be applied to any encoder-decoder IC system. They are applied at test time once the IC system is trained. In particular, we propose to rectify a sentence once it is fully generated (post-generation strategy) or at each time instant of the generation process (in-generation strategy). Experiments conducted on four different IC datasets confirm the promising capabilities of the proposed postprocessing strategies to rectify the output of a simple encoder-decoder by generating more coherent descriptions. The achieved results are competitive and sometimes better than complex IC systems.
Keyword:
Convolutional neural networks (CNNs)
hidden Markov models (HMMs)
image captioning (IC)
recurrent neural networks (RNNs)
unmanned aerial vehicles (UAVs)
Viterbi algorithm
期刊
IF:
8.6
论文数:
2.1W
被引数:
10.7W
机构
引用论文
A TUTORIAL ON HIDDEN MARKOV-MODELS AND SELECTED APPLICATIONS IN SPEECH RECOGNITION关于语音识别中的隐马尔可夫模型和选定应用的教程.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
分享收藏分享收藏Exploring Transformer and Multilabel Classification for Remote Sensing Image Captioning探索用于遥感图像字幕的变压器和多标签分类
OAAI
OAAI分享收藏Do Perceptions of Competence Mediate The Relationship Between Fundamental Motor Skill Proficiency and Physical Activity Levels of Children in Kindergarten?能力的感知是否可以介导幼儿园儿童的基本运动技能熟练程度与身体活动水平之间的关系？
PREAI
分享收藏Nanostructured hybrid NiFeOOH/CNT electrocatalysts for oxygen evolution reaction with low overpotentialRSC Advances
RSC AdvancesIF0
PREAI
分享收藏
查看更多内容