.png){kind=icon}
返回
CLIP guided image caption decoding based on monte carlo tree search
.png){kind=icon}
PRE
AI
DOI：10.1007/s00530-024-01656-7.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, image captioning has attracted significant research interest due to its wide range of applications. The training process for captioning generative models typically employs backpropagation to maximize the likelihood of the next word given the previous ground truth word. Evaluation of this method is usually conducted using metrics such as CIDEr, ROUGE, and others. However, a disadvantage of this approach is that the words sampled by the network must closely resemble the ground truth. Since the word generation process is not continuous, it is challenging to optimize these metrics directly through backpropagation. In this study, we propose a novel approach to image caption decoding that enhances the robustness of existing models by incorporating prior knowledge from pre-trained cross-modal CLIP models. The results of comparative experiments demonstrate that the Monte Carlo Tree Search (MCTS)-based approach yields favorable outcomes in terms of evaluation metrics.
Keyword:
Image caption
Ground-truth
MCTS
Cross-modality
CLIP
期刊
IF:
3.1
论文数:
2.8K
被引数:
2.7K
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏
分享收藏
查看更多内容