.png){kind=icon}
返回
Robust Validation: Confident Predictions Even When Distributions Shift
.png){kind=icon}
PRE
AI
DOI：10.1080/01621459.2023.2298037.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
While the traditional viewpoint in machine learning and statistics assumes training and testing samples come from the same population, practice belies this fiction. One strategy-coming from robust statistics and optimization-is thus to build a model robust to distributional perturbations. In this article, we take a different approach to describe procedures for robust predictive inference, where a model provides uncertainty estimates on its predictions rather than point predictions. We present a method that produces prediction sets (almost exactly) giving the right coverage level for any test distribution in an f-divergence ball around the training population. The method, based on conformal inference, achieves (nearly) valid coverage in finite samples, under only the condition that the training data be exchangeable. An essential component of our methodology is to estimate the amount of expected future data shift and build robustness to it; we develop estimators and prove their consistency for protection and validity of uncertainty estimates under shifts. By experimenting on several large-scale benchmark datasets, including Recht et al.'s CIFAR-v4 and ImageNet-V2 datasets, we provide complementary empirical results that highlight the importance of robust predictive validity. Supplementary materials for this article are available online.
Keyword:
Confidence sets
Conformal inference
Coverage validity
f-divergences
Robust statistics
期刊
J
IF:
3
论文数:
5.2K
被引数:
4.8W
机构
引用论文
分享收藏
LEARNING MODELS WITH UNIFORM PERFORMANCE VIA DISTRIBUTIONALLY ROBUST OPTIMIZATION通过分布式鲁棒优化学习具有统一性能的模型.png)
ANNALS OF STATISTICS
OAAI
ANNALS OF STATISTICSIF3.7
OAAI分享收藏
查看更多内容