.png){kind=icon}
返回
Accelerating hyperparameter optimization with a secretary
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.129455.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Hyperparameter optimization (HPO) is a vital step in machine learning (ML) for enhancing model performance. However, the vast and complex nature of the search space makes HPO both challenging and resource- intensive. Automatic HPO methods have demonstrated their ability to efficiently explore high-dimensional hyperparameter spaces and identify optimal solutions, but training and evaluating the model for each set of hyperparameters is still computationally expensive. To further reduce the computational cost, we propose a novel strategy that wraps the HPO process and terminates it based on the sequence of hyperparameters evaluated. The algorithm is inspired by the classic secretary problem, with two additional variations to better adjust to the HPO process. We evaluated the algorithm using popular HPO samplers, including Random Search (RS), Tree-structured Parzen Estimator (TPE), Bayesian Optimization with Gaussian Processes (BOGP), Genetic Algorithms (GA), and Particle Swarm Optimization (PSO). Results indicate that the proposed algorithm accelerates the HPO process by an average of 34%, with only a minimal trade-off in solution quality of 8%. The algorithm is straightforward to implement, compatible with any HPO setup, and particularly effective in the early stages of optimization. This makes it a valuable tool for practitioners aiming to quickly identify promising hyperparameters or reducing the search space, significantly cutting down the time and computational resources required.
Keyword:
Hyperparameter optimization
Secretary problem
Machine learning
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
The rise of the mediocre forest: why chronically stressed trees may better survive extreme episodic climate variability.png)
New Forests
New ForestsIF0
PREAI
分享收藏Analysis and assessment of ship collision accidents using Fault Tree and Multiple Correspondence Analysis基于故障树和多重对应分析的船舶碰撞事故分析与评估
PREAI
分享收藏Improving classification accuracy of fine-tuned CNN models: Impact of hyperparameter optimization提高微调CNN模型的分类精度: 超参数优化的影响HELIYON
OAAI
HELIYONIF3.6
OAAI分享收藏On hyperparameter optimization of machine learning algorithms: Theory and practice机器学习算法的超参数优化: 理论与实践NEUROCOMPUTINGOAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏Hyperparameter Tuning for Machine Learning Algorithms Used for Arabic Sentiment Analysis用于阿拉伯语情感分析的机器学习算法的超参数调整INFORMATICS-BASEL
OAAI
INFORMATICS-BASELIF2.8
OAAI分享收藏SciPy 1.0: fundamental algorithms for scientific computing in PythonSciPy 1.0: Python中科学计算的基本算法NATURE METHODSOAAI
NATURE METHODSIF32.1
OAAI分享收藏Robust autonomous vehicle control by leveraging multi-stage MPC and quantized CNN in HIL Framework在HIL框架中利用多级MPC和量化CNN的鲁棒自主车辆控制
PREAI
分享收藏
查看更多内容