TESTING FOR OUTLIERS WITH CONFORMAL P-VALUES

Stephen Bates

doi:10.1214/22-AOS2244

TESTING FOR OUTLIERS WITH CONFORMAL P-VALUES

2023-02-01

36

OA

AI

DOI：10.1214/22-AOS2244

Original

Original request for help

Save

Abstract

En 中文

This paper studies the construction of p-values for nonparametric out-lier detection, from a multiple-testing perspective. The goal is to test whether new independent samples belong to the same distribution as a reference data set or are outliers. We propose a solution based on conformal inference, a general framework yielding p-values that are marginally valid but mutually dependent for different test points. We prove these p-values are positively de-pendent and enable exact false discovery rate control, although in a relatively weak marginal sense. We then introduce a new method to compute p-values that are valid conditionally on the training data and independent of each other for different test points; this paves the way to stronger type-I error guarantees. Our results depart from classical conformal inference as we leverage con-centration inequalities rather than combinatorial arguments to establish our finite-sample guarantees. Further, our techniques also yield a uniform confi-dence bound for the false positive rate of any outlier detection algorithm, as a function of the threshold applied to its raw statistics. Finally, the relevance of our results is demonstrated by experiments on real and simulated data.

Keywords:

Conformal inference

out-of-distribution

false discovery rate

positive dependence

TESTING FOR OUTLIERS WITH CONFORMAL P-VALUES

Abstract

Journal

Organization