.png){kind=icon}
.png){kind=icon}
Return
HyperSTAR: Task-Aware Hyperparameter Recommendation for Training and Compression
.png){kind=icon}
PRE
AI
DOI：10.1007/s11263-023-01961-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Hyperparameter optimization (HPO) methods alleviate the significant effort required to obtain hyperparameters that perform optimally on visual learning problems. Existing methods are computationally inefficient because they are task agnostic (i.e., they do not adapt to a given task). We present HyperSTAR (System for Task Aware Hyperparameter Recommendation), a task-aware HPO algorithm that improves HPO efficiency for a target dataset by using prior knowledge from previous hyperparameter searches to recommend effective hyperparameters conditioned on the target dataset. HyperSTAR ranks and recommends hyperparameters by predicting their performance on the target dataset. To do so, it learns a joint datasethyperparameter space in an end-to-end manner that enables its performance predictor to use previously found effective hyperparameters for other similar tasks. The hyperparameter recommendations of HyperSTAR combined with existing HPO techniques lead to a task-aware HPO system that reduces the time to find the optimal hyperparameters for the target learning problem. Our experiments on image classification, object detection, and model pruning validate that HyperSTAR reduces the evaluation of different hyperparameter configurations by about 50% compared to existing methods and, when combined with Hyperband, uses only 25% of the budget required by the vanilla Hyperband and Bayesian Optimized Hyperband to achieve the best performance.
Keywords:
Hyperparameter search
AutoML
Warm start
Meta-learning
Image classification
Object detection
Model pruning
Journal
IF:
9.3
Papers:
3.9K
Citations:
2.8W