ℓ1-regularization learning based on Huber regression

doi:10.1080/02331888.2025.2597296

Return

ℓ1-regularization learning based on Huber regression

2025-12-01

PRE

Xiao Zhang

许鸿飞 (Hongfei Xu)

ShouYou HUANG *

DOI：10.1080/02331888.2025.2597296

Original

Original request for help

Save

Abstract

En 中文

Building robust models that remain stable under outliers and heavy-tailed noise is a central challenge in high-dimensional data analysis. To address this issue, this paper proposes a kernel-based $ \ell _1 $ & ell;1-regularized Huber regression framework to enhance robustness against non-Gaussian disturbances. The approach integrates the inherent resistance of the Huber loss function to outliers with $ \ell _1 $ & ell;1-regularization, thereby improving generalization and stability. Theoretically, within the reproducing kernel Hilbert space (RKHS), we derive explicit upper bounds for the approximation, sample, and excess errors, along with learning rates that guarantee convergence and robustness. The proposed method is empirically evaluated on both simulated and real-world datasets. Compared with OLS, Ridge, and Lasso methods, the method demonstrates superior robustness under varying levels of noise. In experiments across five Kaggle datasets, it consistently achieves the lowest RMSE. Overall, the proposed method exhibits strong theoretical guarantees and empirical robustness, providing a reliable framework for learning in high-dimensional and noisy environments.

Keywords:

Coefficient regularization

Huber loss

learning rate

error decomposition