.png){kind=icon}
.png){kind=icon}
Return
Comprehensive gradient-free optimization plugin with kernel density estimation-based cyclical learning rate and dynamic bandwidth adaptation
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2025.126519.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Traditional gradient optimization methods often face performance bottlenecks in complex non-convex tasks, multimodal data, and high-dimensional parameter spaces, struggling to provide robust performance across diverse application scenarios. To address these challenges, we propose a Comprehensive Gradient-Free Optimization Plugin with Kernel Density Estimation-Based Cyclical Learning Rate and Dynamic Bandwidth Adaptation (CGFO-KCD), designed to enhance the performance of traditional optimizers. CGFO-KCD introduces gradient-free optimization strategies and a heuristic bandwidth adjustment mechanism, utilizing prior gradient norms and parameter variance from memory space to guide the sampling direction and magnitude in Kernel Density Estimation (KDE). This enables adaptive sampling in the parameter space, significantly improving the optimizer's global search capability. Additionally, the plugin employs Cyclical Learning Rate (CLR) scheduling to adaptively adjust the learning rate at different optimization stages, enhancing both efficiency and stability. A series of rigorous experiments, including benchmark performance evaluations across three typical application scenarios, generalization verification on industrial application scenarios, ablation studies, and parameter sensitivity analyses, validate the effectiveness of CGFO-KCD. The results demonstrate that CGFO-KCD significantly enhances the performance of traditional optimizers across various application scenarios, providing strong support and innovative solutions for complex optimization challenges without excessive parameter dependency.
Keywords:
Gradient-free optimization
Kernel density estimation (KDE)
Cyclical learning rate (CLR)
Heuristic strategy
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W
Organization
No organization information available