Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

Guangyong Chen

doi:10.1109/TAC.2024.3376315

Return

Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

2024-09-01

PRE

陈晨 (C. L. Philip Chen)

DOI：10.1109/TAC.2024.3376315

Original

Original request for help

Save

Abstract

En 中文

Robust nonlinear regression frequently arises in data analysis that is affected by outliers in various application fields such as system identification, signal processing, and machine learning. However, it is still quite challenge to design an efficient algorithm for such problems due to the nonlinearity and nonsmoothness. Previous researches usually ignore the underlying structure presenting in the such nonlinear regression models, where the variables can be partitioned into a linear part and a nonlinear part. Inspired by the high efficiency of variable projection algorithm for solving separable nonlinear least squares problems, in this article, we develop a robust variable projection (RoVP) method for the parameter estimation of separable nonlinear regression problem with L-1 norm loss. The proposed algorithm eliminates the linear parameters by solving a linear programming subproblem, resulting in a reduced problem that only involves nonlinear parameters. More importantly, we derive the Jacobian matrix of the reduced objective function, which tackles the coupling between the linear parameters and nonlinear parameters. Furthermore, we observed an intriguing phenomenon in the landscape of the original separable nonlinear objective function, where some narrow valleys frequently exist. The RoVP strategy can effectively diminish the likelihood of the algorithm getting stuck in these valleys and accelerate its convergence. Numerical experiments confirm the effectiveness and robustness of the proposed algorithm.

Keywords:

Optimization

Jacobian matrices

Linear programming

Parameter estimation

Autoregressive processes

Signal processing algorithms

Predictive models

Radial basis function network based autoregressive (RBF-AR) model

robust parameter estimation

system identification

variable projection (VP) algorithm