Accelerated Proximal Subsampled Newton Method

Haishan Ye

doi:10.1109/TNNLS.2020.3017555

Accelerated Proximal Subsampled Newton Method

2021-10-01

4

PRE

AI

DOI：10.1109/TNNLS.2020.3017555

原文链接

原文求助

摘要

En 中文

Composite function optimization problem often arises in machine learning known as regularized empirical minimization. We introduce the acceleration technique to the Newton-type proximal method and propose a novel algorithm called accelerated proximal subsampled Newton method (APSSN). APSSN only subsamples a small subset of samples to construct an approximate Hessian that achieves computational efficiency. At the same time, APSSN still keeps a fast convergence rate. Furthermore, we obtain the scaled proximal mapping by solving its dual problem using the semismooth Newton method instead of resorting to the first-order methods. Due to our sampling strategy and the fast convergence rate of the semismooth Newton method, we can get the scaled proximal mapping efficiently. Both our theoretical analysis and empirical study show that APSSN is an effective and computationally efficient algorithm for composite function optimization problems.

Keyword:

Convergence

Acceleration

Approximation algorithms

Newton method

Optimization

Machine learning

Stochastic processes

Approximate Newton

Nesterov's acceleration

nonsmooth optimization

proximal algorithm

Accelerated Proximal Subsampled Newton Method

摘要

AI总结

期刊

机构

引用论文