Natural Language Processing Versus Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation

Chengyi Zheng

doi:10.2196/57949

Natural Language Processing Versus Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation

2024-09-10

0

OA

AI

DOI：10.2196/57949

查看原文

摘要

En 中文

Background: Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), adebilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manualchart review is sometimes used to identify PHN in electronic health records (EHRs), which can be costly and time-consuming. Objective: This study aims to develop and validate a natural language processing (NLP) algorithm for automaticallyidentifying PHN from unstructured EHR data and to compare its performance with that of code-based methods. Methods: This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated healthcare system that serves over 4.8 million members. The source population included members aged >= 50 years who receivedan incident HZ diagnosis and accompanying antiviral prescription between 2018 and 2020 and had >= 1 encounter within90-180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. ForNLP development and validation, 500 and 800 random samples from the source population were selected, respectively. Thesensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlationcoefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. Results: The NLP algorithm identified PHN cases with a 90.9% sensitivity, 98.5% specificity, 82% PPV, and 99.3% NPV.The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validationdata were 6.9% (reference standard), 7.6% (NLP), and 5.4%-13.1% (code-based). The code-based methods achieved a52.7%-61.8% sensitivity, 89.8%-98.4% specificity, 27.6%-72.1% PPV, and 96.3%-97.1% NPV. The F-scores and MCCsranged between 0.45 and 0.59 and between 0.32 and 0.61, respectively. Conclusions: The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This methodcould be useful in population-based PHN research

Keyword:

postherpetic neuralgia

herpes zoster

natural language processing

electronic health record

real-world data

artificial intelligence

development

validation

diagnosis

EHR

algorithm

EHR data

sensitivity

specificity

validation data

neuralgia

recombinant zoster vaccine

Natural Language Processing Versus Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation

摘要

AI总结

期刊

机构

引用论文