.png){kind=icon}
.png){kind=icon}
Return
A Single-Loop Algorithm for Decentralized Bilevel Optimization
.png){kind=icon}
PRE
AI
DOI：10.1287/moor.2024.0488.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Bilevel optimization has gained significant attention in recent years because of its broad applications in machine learning. This paper focuses on bilevel optimization in decentralized networks and proposes a novel single-loop algorithm for solving decentralized bilevel optimization with a strongly convex lower-level problem. Our approach is built on the basis of the SOBA framework, and it is a fully single-loop method that approximates the hypergradient by using merely two matrix-vector multiplications per iteration. Importantly, by incorporating the gradient tracking and projection techniques, our algorithm does not require any gradient heterogeneity assumption, which distinguishes it from existing methods for decentralized bilevel optimization and federated bilevel optimization. We establish the convergence rate of the proposed algorithm. Moreover, we present experimental results on hyperparameter optimization and data hyper-cleaning problems, which demonstrate the efficiency of our proposed algorithm.
Keywords:
decentralized optimization
bilevel optimization
hyperparameter tuning
iteration complexity
heterogeneity
Journal
M
IF:
1.9
Papers:
77
Citations:
0
View more organizations