.png){kind=icon}
返回
Interactive Image Caption Generation Reflecting User Intent from Trace Using a Diffusion Language Model
.png){kind=icon}
PRE
AI
DOI：10.20965/jaciii.2025.p1417.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This study proposes an image captioning method designed to incorporate user-specific explanatory intentions into the generated text, as signaled by the user's trace on the image. We extract areas of interest from dense sections of the trace, determine the order of explanations by tracking changes in the pen-tip coordinates, and assess the degree of interest in each area by analyzing the time spent on them. Additionally, a diffusion language model is utilized to generate sentences in a non-autoregressive manner, allowing control over sentence length based on the temporal data of the trace. In the actual caption generation task, the proposed method achieved higher string similarity than conventional methods, including autoregressive models, and successfully captured user intent from the trace and faithfully reflected it in the generated text.
Keyword:
deep learning
natural language processing
image captioning
diffusion model
interaction
期刊
J
IF:
0.8
论文数:
87
被引数:
626
机构
引用论文
.png){kind=icon}
分享收藏
查看更多内容