.png){kind=icon}
返回
Mini-batch algorithms with Barzilai-Borwein update step
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2018.06.002.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
As a way to accelerate stochastic schemes, mini-batch optimization has been a popular choice for large scale learning due to its good general performance and ease of parallel computing. However, the performance of mini-batch algorithms can vary significantly based on the choice of the step size sequence, and, in general, there is a paucity of guidance for making good choices. In this paper, we propose to use the Barzilai-Borwein (BB) update step to automatically compute step sizes for the state of the art mini-batch method (mini-batch semi-stochastic gradient descent (mS2GD) method), thereby obtaining a new optimization method: mS2GD-BB. We prove that mS2GD-BB converges linearly in expectation for non-smooth strongly convex objective functions. We analyze the complexity of mS2GD-BB and show that it achieves as fast a rate as modern stochastic gradient methods. Numerical experiments on standard data sets indicate that the performance of mS2GD-BB is superior to some state of the art methods. (C) 2018 Elsevier B.V. All rights reserved.
Keyword:
Stochastic optimization
Mini batches
Barzilai-Borwein method
Variance reduction
Convex optimization
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
.png){kind=icon}
分享收藏
Graphene/Ionic Liquid Binary Electrode Material for High Performance Supercapacitor用于高性能超级电容器的石墨烯/离子液体二元电极材料
PREAI
分享收藏
查看更多内容