.png){kind=icon}
.png){kind=icon}
Return
A Mini-Batch Algorithm with Adaptive Learning Rate Strategy
.png){kind=icon}
PRE
AI
DOI：10.17576/jsm-2026-5503-11.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
To address the limitations of manually selecting step sizes or using diminishing step size sequences, which can slow convergence in mini-batch algorithms, we propose a strategy for automatically calculating step sizes by employing the Positive Defined Stabilized Barzilai-Borwein (PDSBB) method. The PDSBB step size is integrated into the mini-batch semi-stochastic gradient descent (mS2GD) algorithm, creating a novel algorithm called mS2GD-PDSBB. Based on the linear convergence result, the computational complexity is characterized in terms of the expected number of stochastic gradient evaluations required to achieve a prescribed accuracy level. Computational experiments on benchmark instances are conducted to evaluate the convergence behavior of the proposed algorithm. Suitable mini-batch size leads the mS2GD-PDSBB algorithm to successfully attain the performance consistent to the base algorithms. The numerical experiments demonstrate that the proposed mS2GD-PDSBB algorithm achieves stable and fast convergence with the adaptive step-size strategy. In particular, the algorithm shows reduced sensitivity to the choice of initial step sizes and consistently outperforms or matches mS2GD and mS2GD-BB in terms of objective sub-optimality and test error across different dataset.
Keywords:
Adaptive step size
convergence rate
mS2GD algorithm
PDSBB method
Journal
IF:
0.8
Papers:
113
Citations:
2.9K