Population-Based Hyperparameter Tuning With Multitask Collaboration

Wendi Li

doi:10.1109/TNNLS.2021.3130896

Return

Population-Based Hyperparameter Tuning With Multitask Collaboration

2023-09-01

PRE

Wendi Li

王婷 (Ting Wang) *

Wing W. Y. Ng

DOI：10.1109/TNNLS.2021.3130896

Original

Original request for help

Save

Abstract

En 中文

Population-based optimization methods are widely used for hyperparameter (HP) tuning for a given specific task. In this work, we propose the population-based hyperparameter tuning with multitask collaboration (PHTMC), which is a general multitask collaborative framework with parallel and sequential phases for population-based HP tuning methods. In the parallel HP tuning phase, a shared population for all tasks is kept and the intertask relatedness is considered to both yield a better generalization ability and avoid data bias to a single task. In the sequential HP tuning phase, a surrogate model is built for each new-added task so that the metainformation from the existing tasks can be extracted and used to help the initialization for the new task. Experimental results show significant improvements in generalization abilities yielded by neural networks trained using the PHTMC and better performances achieved by multitask metalearning. Moreover, a visualization of the solution distribution and the autoencoder's reconstruction of both the PHTMC and a single-task population-based HP tuning method is compared to analyze the property with the multitask collaboration.

Keywords:

Task analysis

Tuning

Collaboration

Statistics

Sociology

Optimization

Deep learning

Deep neural network

hyperparameter (HP) tuning

multitask learning