.png){kind=icon}
返回
Hyperparameter Optimization Using Iterative Decision Tree (IDT)
.png){kind=icon}
PRE
AI
DOI：10.1109/ACCESS.2022.3212387.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Machine learning and deep learning have gained a lot of attention from researchers because of their promising predictive performance and the availability of extensive high-dimensional data and high-performance computational hardware. However, the performance of these algorithms is susceptible to the choice of hyperparameters, while optimizing these hyperparameters is usually computationally expensive. For this reason, this study proposed a novel sequential search algorithm using decision tree regression as the surrogate function. In each iteration of this algorithm, several new combinations of hyperparameters were selected from a few best-performed leaves of the decision tree, called Iterative Decision Tree (IDT). Our approach could reduce the computational time from repetitive training the surrogate function compared to conventional sequential search algorithms and enable parallel computing. To confirm the effectiveness of the proposed algorithm, it was compared with six popular benchmark optimization algorithms, including Grid Search, Random Search, Bayesian Optimization, Random Forest, Tree-Structured Parzen Estimation, and Genetic Algorithm. The comparison was examined by optimizing three benchmark nonconvex functions, and hyperparameter tuning of two machine learning algorithms (Support Vector Machine and Random Forest) and two deep learning models (Autoencoder and Convolutional Neural Networks). As a result, the proposed algorithm achieved competitive performance with high stability in addition to the feature importance metrics.
Keyword:
Optimization
Machine learning algorithms
Linear programming
Training
Genetic algorithms
Deep learning
Search problems
Decision tree
deep learning
feature importance
hyperparameter optimization
machine learning
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Analysis and assessment of ship collision accidents using Fault Tree and Multiple Correspondence Analysis基于故障树和多重对应分析的船舶碰撞事故分析与评估
PREAI
分享收藏Taking the Human Out of the Loop: A Review of Bayesian Optimization将人类带出循环: 贝叶斯优化的回顾.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏分享收藏Pemanfaatan kulit singkong sebagai bahan baku pembuatan Natrium Karbosimetil Selulosa木薯皮作为制备Natrium Karbosimetil Selulosa的原料利用
OAAI
OAAI分享收藏Optimizing hyperparameters of deep learning in predicting bus passengers based on simulated annealing基于模拟退火的公交乘客预测深度学习超参数优化
PREAI
分享收藏Long Short Term Memory Hyperparameter Optimization for a Neural Network Based Emotion Recognition FrameworkIEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏
查看更多内容