.png){kind=icon}
返回
Regularized online exponentially concave optimization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2024.127789.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this paper, we investigate regularized online exponentially concave (abbr. exp-concave) optimization, in which each loss function consists of a time -varying exp-concave function and a fixed convex regularization. If the whole loss function is exp-concave, a classical method called online Newton step (ONS) enjoys an O ( d log T ) regret bound, where d is the dimensionality and T is the time horizon. However, in the regularized setting, the sum of an exp-concave function and a convex regularization is not necessarily an exp-concave function, which implies that ONS is not applicable. To address this problem, we propose the proximal online Newton step (ProxONS), and show that it can attain the same O ( d log T ) regret bound for any convex regularization. The main idea is to first perform an iteration of ONS with the exp-concave part in each loss function and then perform a proximal mapping with the regularization part. Furthermore, we demonstrate that by utilizing the standard online -to -batch conversion, our ProxONS can be extended to solve stochastic optimization with a regularized exp-concave objective, and enjoy an O ( d log T / T ) convergence rate with high probability. Experimental results on two real datasets verify the effectiveness of our ProxONS.
Keyword:
Online learning
Exponential concavity
Regularization
Proximal mapping
Regret bound
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
分享收藏Incremental on-line learning: A review and comparison of state of the art algorithms.png)
NEUROCOMPUTING
OAAI
NEUROCOMPUTINGIF6.5
OAAI分享收藏
查看更多内容