Data augmented large language models for medical record generation

doi:10.1007/s10489-024-05934-9

Data augmented large language models for medical record generation

2024-12-06

0

PRE

AI

DOI：10.1007/s10489-024-05934-9

原文链接

原文求助

摘要

En 中文

Writing various medical records takes significant daily workload for physicians. Generative AI technique has the advantage in tasks of data-to-text generation and text summarization, and brings opportunities to reduce workload for physicians to work on medical records. However, current general Large Language Models (LLMs) cannot satisfy the strict requirements to correctness of generative texts in specific tasks of medical record generation. In addition, due to the constraints to protect patient privacy, physicians cannot upload patient data to public cloud services for LLM cloud service. We develop optimized LLMs for medical record generation, which can be deployed in hospitals and integrated with the Electronic Medical Record (EMR) applications for physicians to reduce workload of writing medical records. We propose an approach for constructing data augmented LLM on medical record generation. As for each specific task, we extract annotated data with high quality from the EMR application in a hospital. Based on such data and customized instruct, we construct certain optimized models for specific tasks, including medical Data-to-Text generation (from structural medical data to history of present illness) and medical text summarization (from a series of progress notes to discharge summary). Furthermore, we propose Faithfulness score, a evaluation metrics, based on semantic similarity between the generative texts by LLMs and reference texts by physicians. Extensive experiments are conducted with high-quality task-specific medical data, and tested with our optimized models and two other models, including a general state-of-the-art (SOTA) model and a medical model, thereby evaluating the correctness of the generated medical records by Faithfulness score, separately on the two specific tasks.The experimental results demonstrate that our optimized models has improved the Faithfulness score of the generated medical records, respectively by 19.72% and 19.33% rather than the existing SOTA models, on medical Data-to-Text generation and medical text summarization.Our work has been validated and applied in the hospital we cooperate with, and save approximately 0.5-1 hour of working time per day for a physician, so that he or she can spend more time in taking care of his or her patients.This method can be generalized to any hospital, using its native medical data, to achieve a specialized model available for medical record generation tasks.The code is available at https://github.com/LotusPhilip/data-augmented-model

Keyword:

LLM

Medical record generation

Hallucination

Model faithfulness

Chinese embedding model

Data augmented large language models for medical record generation

摘要

期刊

机构

引用论文