From Penalization to Over-Parameterization: Achieving Implicit Regularization for High-Dimensional Linear Errors-in-Variables Models

Jingxuan Luo

doi:10.1080/07350015.2025.2583457

Return

From Penalization to Over-Parameterization: Achieving Implicit Regularization for High-Dimensional Linear Errors-in-Variables Models

2025-11-01

PRE

Jingxuan Luo

Yinfei Kong

Gaorong Li *

DOI：10.1080/07350015.2025.2583457

Original

Original request for help

Save

Abstract

En 中文

Regularization methods are crucial for the analysis of high-dimensional data, with most methods adding a penalty term to the loss function explicitly. In this paper, we introduce an implicit regularization technique through over-parameterization and propose a calibrated penalty-free (CPF) estimation method for high-dimensional linear errors-in-variables models. This method calibrates the bias caused by measurement errors while avoiding the bias introduced by penalty terms. We use the gradient descent algorithm to minimize the over-parameterized calibrated loss function without penalties, resulting in a sparse estimate of regression coefficients and thus achieving implicit regularization. Furthermore, a novel bootstrap methodology is introduced to address the challenge posed by the unknown covariance matrix of measurement errors. The oracle inequality for estimation error is established under certain regularity conditions. Extensive simulation studies and a real data analysis illustrate the competitive finite sample performance of the proposed method.

Keywords:

Errors-in-variables

Gradient descent algorithm

High-dimensional regression

Implicit regularization

Over-parameterization