.png){kind=icon}
返回
A Robust Framework for Distributional Shift Detection Under Sample-Bias
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3393296.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep Neural Networks have been shown to perform poorly or even fail altogether when deployed in real-world settings, despite exhibiting excellent performance on initial benchmarks. This typically occurs due to relative changes in the nature of the production data, often referred to as distributional shifts. In an attempt to increase the transparency, trustworthiness, and overall utility of deep learning systems, a growing body of work has been dedicated to developing distributional shift detectors. As part of our work, we investigate distributional shift detectors that utilize statistical tests of neural network-based representations of data. We show that these methods are prone to fail under sample-bias, which we argue is unavoidable in most practical machine learning systems. To mitigate this, we implement a novel distributional shift detection framework which explicitly accounts for sample-bias via a simple sample-selection procedure. In particular, we show that the effect of sample-bias can be significantly reduced by performing statistical tests against the most similar data in the training set, as opposed to the training set as a whole. We find that this improves the stability and accuracy of a variety of distributional shift detection methods on both covariate- and semantic-shifts, with improvements to balanced accuracy typically ranging between 0.1 and 0.2, and false-positive-rates often being eliminated altogether under bias.
Keyword:
Training
Data models
Training data
Detectors
Deep learning
System performance
Distributed databases
Detection algorithms
system support
neural networks
distributional shift
OOD detection
sample bias
statistical tests
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
分享收藏
WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians在结肠镜检查中准确突出显示息肉的wm-dova地图: 医生的验证与显著性地图
PREAI
分享收藏
查看更多内容