.png){kind=icon}
返回
Annealing Optimization for Progressive Learning With Stochastic Approximation
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2022.3232706.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this work, we introduce a learning model designed to meet the needs of applications in which computational resources are limited, and robustness and interpretability are prioritized. Learning problems can be formulated as constrained stochastic optimization problems, with the constraints originating mainly from model assumptions that define a tradeoff between complexity and performance. This tradeoff is closely related to overfitting, generalization capacity, and robustness to noise and adversarial attacks, and depends on both the structure and complexity of the model, as well as the properties of the optimization methods used. We develop an online prototype-based learning algorithm based on annealing optimization that is formulated as an online gradient-free stochastic approximation algorithm. The learning model can be viewed as an interpretable and progressively growing competitive-learning neural network model to be used for supervised, unsupervised, and reinforcement learning. The annealing nature of the algorithm contributes to minimal hyperparameter tuning requirements, poor local minima prevention, and robustness with respect to the initial conditions. At the same time, it provides online control over the performance-complexity tradeoff by progressively increasing the complexity of the learning model as needed, through an intuitive bifurcation phenomenon. Finally, the use of stochastic approximation enables the study of the convergence of the learning algorithm through mathematical tools from dynamical systems and control, and allows for its integration with reinforcement learning algorithms, constructing an adaptive state-action aggregation scheme.
Keyword:
Approximation algorithms
Optimization
Heuristic algorithms
Annealing
Machine learning algorithms
Dynamical systems
Complexity theory
Optimization for machine learning
annealing optimization
online deterministic annealing (ODA)
progressive learning
reinforcement learning (RL)
stochastic approximation
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
.png){kind=icon}
分享收藏Welding characteristics of aluminum, copper, nickel and aluminum alloy with alumina coating using ultrasonic complex vibration welding equipments铝、铜、镍及铝合金氧化铝涂层超声复合振动焊接特性研究
PREAI
分享收藏分享收藏
查看更多内容