Utilizing Large Language Models for rare and complex diagnoses

doi:10.1016/j.smhl.2026.100658

Utilizing Large Language Models for rare and complex diagnoses

2026-04-01

0

PRE

AI

DOI：10.1016/j.smhl.2026.100658

Original

Original request for help

Save

Abstract

En 中文

The precise diagnosis of rare and complicated diseases is still a major challenge in healthcare of the modern era. Conventional diagnostic methods, which are generally effective for straightforward cases, may not be efficient for situations that are ambiguous or atypical in nature. Large Language Models (LLMs) have recently evolved to a level where they can have a positive effect on clinical reasoning by their sophisticated natural language understanding capabilities. Consequently, we came up with and tested a diagnostic system that uses transformer-based LLMs that are fine-tuned with Low-Rank Adaptation (LoRA) as well as prompt engineering tailored to the task. We ran experiments with several of our LLMs: Falcon 7B, Open Llama 3B v2, and Medical Llama 7B on the Diagnostic Case Challenge Collection (DC3) and parts of MIMIC-III and MIMIC-IV datasets. The findings indicate that our domain-specific models have attained a 53.3% total accuracy rate which is better than GPT-4 (50%) and Bard (47%). Additionally, our models achieved 100% correct identifications of the cases that were misdiagnosed most often, pointing to the highest level of generalizability in complex diagnostic scenarios.

Keywords:

Diagnosis of rare cases

Medical LLMs

Question answering

Prompt engineering

Utilizing Large Language Models for rare and complex diagnoses

Abstract

Journal

Organization