.png){kind=icon}
返回
Deep learning uncertainty quantification for clinical text classification
.png){kind=icon}
OA
AI
DOI：10.1016/j.jbi.2023.104576.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Introduction: Machine learning algorithms are expected to work side-by-side with humans in decision-making pipelines. Thus, the ability of classifiers to make reliable decisions is of paramount importance. Deep neural networks (DNNs) represent the state-of-the-art models to address real-world classification. Although the strength of activation in DNNs is often correlated with the network's confidence, in-depth analyses are needed to establish whether they are well calibrated.Method: In this paper, we demonstrate the use of DNN-based classification tools to benefit cancer registries by automating information extraction of disease at diagnosis and at surgery from electronic text pathology reports from the US National Cancer Institute (NCI) Surveillance, Epidemiology, and End Results (SEER) population based cancer registries. In particular, we introduce multiple methods for selective classification to achieve a target level of accuracy on multiple classification tasks while minimizing the rejection amount-that is, the number of electronic pathology reports for which the model's predictions are unreliable. We evaluate the proposed methods by comparing our approach with the current in-house deep learning-based abstaining classifier. Results: Overall, all the proposed selective classification methods effectively allow for achieving the targeted level of accuracy or higher in a trade-off analysis aimed to minimize the rejection rate. On in-distribution validation and holdout test data, with all the proposed methods, we achieve on all tasks the required target level of accuracy with a lower rejection rate than the deep abstaining classifier (DAC). Interpreting the results for the out-of-distribution test data is more complex; nevertheless, in this case as well, the rejection rate from the best among the proposed methods achieving 97% accuracy or higher is lower than the rejection rate based on the DAC.Conclusions: We show that although both approaches can flag those samples that should be manually reviewed and labeled by human annotators, the newly proposed methods retain a larger fraction and do so without retraining-thus offering a reduced computational cost compared with the in-house deep learning-based abstaining classifier.
Keyword:
Selective classification
Deep learning
Abstaining classifier
Text classification
Uncertainty quantification
Accuracy
DNN
CNN
HiSAN
Pathology reports
NCI SEER
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.5
论文数:
3.5K
被引数:
1.9W
机构
查看更多机构
引用论文
Deep Learning for Automated Extraction of Primary Sites From Cancer Pathology Reports深度学习用于从癌症病理报告中自动提取主要位点OAAI
OAAI分享收藏Second opinion needed: communicating uncertainty in medical machine learning需要第二意见: 在医疗机器学习中传达不确定性.png)
NPJ DIGITAL MEDICINEOAAI
NPJ DIGITAL MEDICINEIF15.1
OAAI分享收藏UV-B effects on the nutritional chemistry of plants and the responses of a mammalian herbivoreOecologia
OecologiaIF0
PREAI
分享收藏Can campaign‐style enforcement work: When and how? Evidence from straw burning control in ChinaGovernance
GovernanceIF0
PREAI
分享收藏 没有更多内容