.png){kind=icon}
.png){kind=icon}
Return
Evaluation of the Mayo endoscopic score in ulcerative colitis using a multimodal large language model: a human-blinded accuracy study
.png){kind=icon}
PRE
AI
R
G
M
I
F
S
A
DOI：10.5217/ir.2025.00138.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Background/Aims: To date, some large language models (LLMs), such as Chat Generative Pre-trained Transformer 4-omni (ChatGPT-4o), can process images through visual transformer patching. An LLM analysis was conducted to assess the ability of ChatGPT-4o to assign the Mayo endoscopic score (MES). Methods: A selection set of high-quality endoscopic frames was identified to compare 4 input models to select the most performant, confirmed in an extended set of 304 frames. Concordance with the evaluation by expert endoscopists was assessed. Results: Only one of the pre-tested models demonstrated significant concordance (kappa = 0.232, 95% confidence interval [CI] = 0.167 to 0.296, P= 0.003; rho = 0.36, P= 0.011), with a mean bias of -0.26 +/- 1.192 (95% CI, -2.596 to 2.076). This was confirmed in the extended set (kappa = 0.260, 95% CI = 0.195 to 0.324, P< 0.001; rho = 0.288, P< 0.001). The absolute concordance for the selected model was 44% and 45.3% in the selection and extended sets, respectively. For the identification of moderately-to-severely active disease, a sensitivity of 73% (95% CI, 60% to 82%), specificity of 60% (95% CI, 54% to 66%), positive predictive value of 32% (95% CI, 25% to 40%), and negative predictive value of 90% (95% CI, 84% to 93%) were identified. Conclusions: ChatGPT-4o shows a mild potential in evaluating MES in endoscopic frames, but further refinements are mandatory.
Keywords:
Generative artificial intelligence
Large language models
Mayo endoscopic score
Colonoscopy
Journal
IF:
3.9
Papers:
371
Citations:
1.6K
Organization
Cited Papers