.png){kind=icon}
.png){kind=icon}
Return
Ascle-A Python Natural Language Processing Toolkit for MedicalText Generation:Development and Evaluation Study
.png){kind=icon}
OA
AI
DOI：10.2196/60601.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Background: Medical texts present significant domain-specific challenges, and manually curating these texts is a time-consumingand labor-intensive process. To address this, natural language processing (NLP) algorithms have been developed to automatetext processing. In the biomedical field, various toolkits for text processing exist, which have greatly improved the efficiency ofhandling unstructured text. However, these existing toolkits tend to emphasize different perspectives, and none of them offergeneration capabilities, leaving a significant gap in the current offerings.Objective: This study aims to describe the development and preliminary evaluation of Ascle. Ascle is tailored for biomedicalresearchers and clinical staff with an easy-to-use, all-in-one solution that requires minimal programming expertise. For the firsttime, Ascle provides 4 advanced and challenging generative functions: question-answering, text summarization, text simplification,and machine translation. In addition, Ascle integrates 12 essential NLP functions, along with query and search capabilities forclinical databases.Methods: We fine-tuned 32 domain-specific language models and evaluated them thoroughly on 27 established benchmarks.In addition, for the question-answering task, we developed a retrieval-augmented generation (RAG) framework for large language models that incorporated a medical knowledge graph with ranking techniques to enhance the reliability of generated answers.Additionally, we conducted a physician validation to assess the quality of generated content beyond automated metrics.Results: The fine-tuned models and RAG framework consistently enhanced text generation tasks. For example, the fine-tunedmodels improved the machine translation task by 20.27 in terms of BLEU score. In the question-answering task, the RAGframework raised the ROUGE-L score by 18% over the vanilla models. Physician validation of generated answers showed highscores for readability (4.95/5) and relevancy (4.43/5), with a lower score for accuracy (3.90/5) and completeness (3.31/5).Conclusions: This study introduces the development and evaluation of Ascle, a user-friendly NLP toolkit designed for medicaltext generation. All code is publicly available through the Ascle GitHub repository. All fine-tuned language models can beaccessed through Hugging Face
Keywords:
natural language processing
machine learning
deep learning
generative artificial intelligence
large language models
retrieval-augmented generation
healthcare
Journal
IF:
6
Papers:
9.7K
Citations:
5.0W
View more organizations