Optimizing Medical Image Captioning with Conditional Prompt Encoding

Fernandes, Rendson F.

doi:10.1007/978-3-031-99568-2_16

2026-01-01

PRE

Fernandes, Rendson F.

Hugo S. Oliveira *

Pedro Ribeiro

Hélder P. Oliveira

DOI：10.1007/978-3-031-99568-2_16

原文链接

原文求助

摘要

En 中文

医学图像字幕生成是产生医学图像描述性文本报告的重要工具。医学图像字幕生成的核心问题之一是其领域描述生成能力较差，因为大型预训练语言模型主要在非医学文本领域进行训练，而医学文本的语义不同。为克服这一局限，我们探索了针对X射线图像的对比学习改进，并结合软提示工程（soft prompt engineering）用于医学图像字幕生成和条件文本解码。主要目标是开发一个软提示模型，以提高自动生成字幕的准确性和临床相关性，同时保证其完整的语言准确性而不损害模型性能。在MIMIC-CXR和ROCO数据集上的实验表明，引入定制的软提示提高了准确性和效率，同时确保了字幕更具连贯的医学背景，有助于医学诊断并鼓励更准确的报告。

Keyword:

Transformers

Contrastive Learning

Soft Prompts

Image Caption

期刊

PATTERN RECOGNITION AND IMAGE ANALYSIS, IBPRIA 2025, PT II

IF:

论文数:

被引数:

机构

universidade do porto

学者数:

4.3K

论文数: 1.7K

被引数: 4

引用论文

PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation

Proceedings of the AAAI Conference on Artificial Intelligence

IF0

2024-03-24

OAAI

Haibo Jin; Haoxuan Che; Yi Lin; Hao Chen

Chest X-Ray Report Generation Through Fine-Grained Label Learning 通过细粒度标签学习生成胸部x光报告

Lecture Notes in Computer Science

IF0

2020-09-29

PREAI

Tanveer Syeda-Mahmood; Ken C. L. Wong; Yaniv Gur; Joy T. Wu; Ashutosh Jadhav; Satyananda Kashyap; Alexandros Karargyris; Anup Pillai; Arjun Sharma; Ali Bin Syed; Orest Boyko; Mehdi Moradi

Multimodal Large Language Models in Health Care:Applications,Challenges, and Future Outlook 医疗保健中的多模态大语言模型: 应用，挑战和未来展望

JOURNAL OF MEDICAL INTERNET RESEARCH

IF6

2024-09-25

OAAI

AlSaad, Rawan; Abd-alrazaq, Alaa; Boughorbel, Sabri; Ahmed, Arfan; Renault, Max-Antoine; Damseh, Rafat; Sheikh, Javaid

TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-Rays

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IF0

2018-06-01