.png){kind=icon}
返回
Implementation of a Large-Scale Image Curation Workflow Using Deep Learning Framework
.png){kind=icon}
OA
AI
DOI：10.1016/j.xops.2022.100198.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Purpose: The curation of images using human resources is time intensive but an essential step for developing artificial intelligence (AI) algorithms. Our goal was to develop and implement an AI algorithm for image curation in a high-volume setting. We also explored AI tools that will assist in deploying a tiered approach, in which the AI model labels images and flags potential mislabels for human review.Design: Implementation of an AI algorithm.Participants: Seven-field stereoscopic images from multiple clinical trials.Methods: The 7-field stereoscopic image protocol includes 7 pairs of images from various parts of the central retina along with images of the anterior part of the eye. All images were labeled for field number by reading center graders. The model output included classification of the retinal images into 8 field numbers. Probability scores (0e1) were generated to identify misclassified images, with 1 indicating a high probability of a correct label.Main Outcome Measures: Agreement of AI prediction with grader classification of field number and the use of probability scores to identify mislabeled images.Results: The AI model was trained and validated on 17 529 images and tested on 3004 images. The pooled agreement of field numbers between grader classification and the AI model was 88.3% (kappa, 0.87). The pooled mean probability score was 0.97 (standard deviation [SD], 0.08) for images for which the graders agreed with the AI-generated labels and 0.77 (SD, 0.19) for images for which the graders disagreed with the AI-generated labels (P < 0.0001). Using receiver operating characteristic curves, a probability score of 0.99 was identified as a cutoff for distinguishing mislabeled images. A tiered workflow using a probability score of < 0.99 as a cutoff would include 27.6% of the 3004 images for human review and reduce the error rate from 11.7% to 1.5%.Conclusions: The implementation of AI algorithms requires measures in addition to model validation. Tools to flag potential errors in the labels generated by AI models will reduce inaccuracies, increase trust in the system, and provide data for continuous model development. Ophthalmology Science 2022;2:100198 & COPY; 2022 by the American Academy of Ophthalmology. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Keyword:
Artificial intelligence
Deep learning
Fundus photograph
Image curation
Machine learning
Metadata
Retinal image
Standardization
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.6
论文数:
1.2K
被引数:
1.6K
机构
引用论文
A global review of publicly available datasets for ophthalmological imaging: barriers to access, usability, and generalisability.png)
LANCET DIGITAL HEALTHOAAI
LANCET DIGITAL HEALTHIF24.1
OAAI分享收藏Gaps in standards for integrating artificial intelligence technologies into ophthalmic practice将人工智能技术整合到眼科实践中的标准差距
OAAI
OAAI分享收藏Grader Variability and the Importance of Reference Standards for Evaluating Machine Learning Models for Diabetic RetinopathyOPHTHALMOLOGYOAAI
OPHTHALMOLOGYIF9.5
OAAI分享收藏分享收藏
查看更多内容