.png){kind=icon}
.png){kind=icon}
Return
Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimization
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2024.3486238.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The increasing adoption of Artificial Intelligence (AI) in engineering problems calls for the development of calibration methods capable of offering robust statistical reliability guarantees. The calibration of black box AI models is carried out via the optimization of hyperparameters dictating architecture, optimization, and/or inference configuration. Prior work has introduced learn-then-test (LTT), a calibration procedure for hyperparameter optimization (HPO) that provides statistical guarantees on average performance measures. Recognizing the importance of controlling risk-aware objectives in engineering contexts, this work introduces a variant of LTT that is designed to provide statistical guarantees on quantiles of a risk measure. We illustrate the practical advantages of this approach by applying the proposed algorithm to a radio access scheduling problem.
Keywords:
Reliability
Calibration
Artificial intelligence
Testing
Optimization
Wireless communication
Signal processing algorithms
Resource management
Hyperparameter optimization
Delays
hyperparameter optimization
risk control
multiple hypothesis testing
quantiles
Journal
IF:
9.6
Papers:
1.1W
Citations:
1.7W