.png){kind=icon}
返回
Single-objective and multi-objective optimization for variance counterbalancing in stochastic learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.asoc.2023.110331.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Artificial neural networks have proved to be useful in a host of demanding applications, there-fore becoming increasingly important in science and engineering. Large-scale problems constitute a challenging task for training neural networks using the stochastic gradient descent method and variations, which are based on the random selection of mini-batches of training points at every iteration. The challenge lies on the mandatory use of diminishing search step sizes in order to retain mild error fluctuations throughout the training set preserving so the quality of the network's generalization capability. Variance counterbalancing was recently proposed as a remedy for addressing the diminishing step sizes in neural network training using stochastic gradient methods. It is based on the concurrent minimization of the average mean squared error of the network along with the error variance over random sets of mini-batches. Also, it promotes the use of advanced optimization algorithms instead of the slowly convergent gradient descent. The present work aims at enriching our understanding of the original variance counterbalancing approach, as well as reformulating it as a multi-objective problem by taking advantage of its bi-objective nature. Experimental analysis reveals the performances of the studied approaches and their competitive edge over the established Adam method.& COPY; 2023 Elsevier B.V. All rights reserved.
Keyword:
Variance counterbalancing
Stochastic learning
Neural networks
Single -objective optimization
Multi -objective optimization
期刊
IF:
6.6
论文数:
1.4W
被引数:
4.8W
机构
引用论文
Job tenure and quality of work life of people with psychiatric disabilities working in social enterprises在社会企业工作的精神病患者的工作任期和工作生活质量
PREAI
分享收藏The Danish fabricated metal industry: A competitive medium-low-tech industry in a highwage country
OAAI
OAAI分享收藏WiFiHonk: Smartphone-Based Beacon Stuffed WiFi Car2X-Communication System for Vulnerable Road User SafetyWiFiHonk: 基于智能手机的信标填充WiFi Car2X-Communication系统，用于脆弱的道路使用者安全
PREAI
分享收藏Visible region absorption in TMDs/phosphorene heterostructures for use in solar energy conversion applications.png)
RSC AdvancesOAAI
RSC AdvancesIF0
OAAI分享收藏
查看更多内容