Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference

doi:10.1109/TSP.2025.3629292

Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference

2025-01-01

0

PRE

AI

DOI：10.1109/TSP.2025.3629292

Original

Original request for help

Save

Abstract

En 中文

The application of artificial intelligence (AI) models in fields such as engineering is limited by the known difficulty of quantifying the reliability of an AI’s decision. A well-calibrated AI model must correctly report its accuracy on in-distribution (ID) inputs, while also enabling the detection of out-of-distribution (OOD) inputs. Conventional solutions are effective at addressing only one of these two conflicting requirements. This paper proposes a novel solution that endows Bayesian learning with enhanced ID calibration and OOD detection capabilities by integrating calibration regularization for improved ID performance, confidence minimization for OOD detection, and selective calibration to ensure a synergistic use of calibration regularization and confidence minimization. Selective calibration rejects inputs for which the calibration performance is expected to be insufficient, supporting effective OOD detection, while also ensuring ID calibration. Prior art had only considered these ideas in isolation and for frequentist learning. Numerical results illustrate the trade-offs between ID accuracy, ID calibration, and OOD calibration, showing that the proposed novel Bayesian approach achieves the best ID and OOD performance compared to existing state-of-the-art approaches, at the cost of rejecting a fraction of the inputs.

Keywords:

Bayesian learning

calibration

OOD detection

selective calibration.

Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference

Abstract

Journal

Organization