.png){kind=icon}
返回
A Single-Loop Algorithm for Decentralized Bilevel Optimization
.png){kind=icon}
PRE
AI
DOI：10.1287/moor.2024.0488.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Bilevel optimization has gained significant attention in recent years because of its broad applications in machine learning. This paper focuses on bilevel optimization in decentralized networks and proposes a novel single-loop algorithm for solving decentralized bilevel optimization with a strongly convex lower-level problem. Our approach is built on the basis of the SOBA framework, and it is a fully single-loop method that approximates the hypergradient by using merely two matrix-vector multiplications per iteration. Importantly, by incorporating the gradient tracking and projection techniques, our algorithm does not require any gradient heterogeneity assumption, which distinguishes it from existing methods for decentralized bilevel optimization and federated bilevel optimization. We establish the convergence rate of the proposed algorithm. Moreover, we present experimental results on hyperparameter optimization and data hyper-cleaning problems, which demonstrate the efficiency of our proposed algorithm.
Keyword:
decentralized optimization
bilevel optimization
hyperparameter tuning
iteration complexity
heterogeneity
期刊
M
IF:
1.9
论文数:
81
被引数:
0
机构
引用论文
分享收藏
Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏
查看更多内容