A context-based meta-reinforcement learning approach to efficient hyperparameter optimization

Xiyuan Liu

doi:10.1016/j.neucom.2021.12.086

A context-based meta-reinforcement learning approach to efficient hyperparameter optimization

2022-03-01

13

PRE

AI

DOI：10.1016/j.neucom.2021.12.086

Original

Original request for help

Save

Abstract

En 中文

ABSTR A C T In this paper, we present a context-based meta-reinforcement learning approach to tackle the challeng-ing data-inefficiency problem of Hyperparameter Optimization (HPO). Specifically, we design an agent which sequentially selects hyperparameters to maximize the expected accuracy of the machine learning algorithm on the validation set. First, we design a context variable that learns the latent embedding of prior experience, and the agent can solve the new tasks efficiently conditioned on it. Second, we employ a multi-task objective method that aims to maximize the average reward across all the meta-training tasks to meta-train the agent. Third, in the adaptation phase, we introduce a quadratic penalty technique to achieve better performance of the agent. Finally, to further improve the efficiency in the adaptation phase, we use a predictive model to evaluate the accuracy of machine learning algorithm instead of train-ing it. We evaluate our approach on 18 real-world datasets and the results demonstrate that our approach outperforms other state-of-the-art optimization methods in terms of test set accuracy and runtime performance. (c) 2022 Elsevier B.V. All rights reserved.

Keywords:

Hyperparameter optimization

Reinforcement learning

Meta-learning

Deep learning

A context-based meta-reinforcement learning approach to efficient hyperparameter optimization

Abstract

Journal

Organization