.png){kind=icon}
Return
prunAdag: an adaptive pruning-aware gradient method
.png){kind=icon}
OA
AI
DOI：10.1007/s10589-025-00723-7.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the relevant/irrelevant approach of Ding et al. (Adv Neural Inf Process Syst 32, 2019) and Zimmer et al. (Mathematical optimization for machine learning: proceedings of the MATH+ thematic Einstein semester 2023, 2025) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient's norm of the form \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal{O}(\log (k)/\sqrt{k+1})$$\end{document}. Numerical experiments on several applications show that it is competitive with existing pruning-aware Frank-Wolfe algorithms, see e.g. Zimmer et al. (Mathematical optimization for machine learning: proceedings of the MATH+ thematic Einstein semester 2023, 2025).
Keywords:
Model pruning
Adaptive first-order methods
Objective-function-free optimisation (OFFO)
Global convergence rate
Journal
C
IF:
2
Papers:
68
Citations:
3.5K
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSaveShareSaveShareSaveShareSaveShareSaveShareSave
View more