.png){kind=icon}
.png){kind=icon}
Return
Optimizing the Accuracy of Natural Language Processing Tools for Pulmonary Embolism Detection Through Integration with Claims Data: The PE-EHR plus Study
.png){kind=icon}
PRE
AI
S
S
D
C
A
M
J
M
E
F
H
H
G
D
M
R
L
G
H
L
B
DOI：10.1055/a-2796-1975.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Background Rule-based natural language processing (NLP) tools can identify pulmonary embolism (PE) via radiology reports. However, their external validity remains uncertain. Methods In this cross-sectional study, 1,712 hospitalized patients (with and without PE) at Mass General Brigham (MGB) hospitals (2016-2021) were analyzed. Two previously published NLP algorithms were applied to radiology reports to identify PE. Chart review by two physicians was the reference standard. We tested three approaches: (A) NLP applied to all patients; (B) NLP limited to radiology reports of patients with principal or secondary International Classification of Diseases 10th revision (ICD-10) PE discharge codes; and (C) NLP applied to patients with PE discharge codes or a Present-on-Admission (POA) indicator (Y) for PE. All others were assumed PE-negative in Approaches B and C to minimize NLP false positives. Weighted estimates were derived from the MGB hospitalized cohort (n 1/4 381,642) to calculate F1 scores (as the harmonic mean of sensitivity and positive predictive value [PPV]). Results In Approach A, both NLP tools showed high sensitivity (82.5%, 93.0%) and specificity (98.9%, 98.7%) but low PPV (60.3%, 59.6%). Approach B improved PPV (95.2%, 94.9%) but reduced sensitivity (74.1%, 76.2%), while Approach C preserved both high sensitivity (82.5%, 93.0%) and PPV (95.6%, 95.8%). Approach C demonstrated the best performance, yielding significantly higher F1 scores for both NLP tools (88.6%, 94.4%) compared with Approach A (69.7%, 72.6%) and Approach B (83.3%, 84.5%) (P < 0.001). Conclusion The accuracy of PE detection improves when rule-based NLP algorithms are operationalized using administrative claims data in addition to radiology reports.
Keywords:
pulmonary embolism
natural language processing
international classification of diseases
electronic health record
accuracy
Journal
IF:
4.3
Papers:
7.0K
Citations:
1.4W
Organization
Cited Papers