.png){kind=icon}
返回
A Stochastic Operator Framework for Optimization and Learning With Sub-Weibull Errors
.png){kind=icon}
OA
AI
DOI：10.1109/TAC.2024.3419186.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This article proposes a framework to study the convergence of stochastic optimization and learning algorithms. The framework is modeled over the different challenges that these algorithms pose, such as 1) the presence of random additive errors (e.g., due to stochastic gradients), and 2) random coordinate updates (e.g., due to asynchrony in distributed set-ups). The article covers both convex and strongly convex problems, and it also analyzes online scenarios, involving changes in the data and costs. This article relies on interpreting stochastic algorithms as the iterated application of stochastic operators, thus allowing us to use the powerful tools of operator theory. In particular, we consider operators characterized by additive errors with sub-Weibull distribution (which parameterize a broad class of errors by their tail probability), and random updates. In this framework, we derive convergence results in mean and high probability, by providing bounds to the distance of the current iteration from a solution of the optimization or learning problem. The contributions are discussed in light of federated learning applications.
Keyword:
Stochastic processes
Convergence
Tail
Optimization
Additives
Random variables
Data models
Federated learning
high probability convergence
inexact optimization
online optimization
stochastic operators
期刊
IF:
7
论文数:
1.3W
被引数:
6.7W
机构
引用论文
.png){kind=icon}
A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization坐标下降原始对偶算法及其在分布式异步优化中的应用
OAAI
OAAI分享收藏SuperMann: A Superlinearly Convergent Algorithm for Finding Fixed Points of Nonexpansive OperatorsSuperMann: 一种寻找非扩张算子不动点的超线性收敛算法OAAI
OAAI分享收藏Emerging Therapies for Brain Metastases in NSCLC, Breast Cancer, and Melanoma: A Critical Review
PREAI
分享收藏Animal venoms as a source of antiviral peptides active against arboviruses: a systematic review
PREAI
分享收藏
查看更多内容