.png){kind=icon}
.png){kind=icon}
Return
Smooth gradient loss: a loss function for gradient regularization in deep learning optimization
.png){kind=icon}
PRE
AI
DOI：10.1007/s11227-025-07954-9.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Loss functions are pivotal in deep learning optimization, directing the model training process by measuring the discrepancy between predicted and actual values. The smooth gradient loss (SGL) function is introduced as an innovative approach to enhance the stability and robustness of deep learning optimization. Traditional loss functions, such as Mean Squared Error (MSE) and Mean Absolute Error (MAE), often exhibit sensitivity to outliers and gradient instability, which can hinder performance in large-scale applications. SGL addresses these issues by incorporating a gradient penalty term, which encourages smoother and more stable gradient updates. Empirical evaluations on benchmark datasets, including MNIST and CIFAR-10, demonstrate that SGL not only accelerates convergence but also enhances generalization performance. Importantly, SGL requires efficient computation of second-order information (Hessian–vector products), which is practical only with high-performance computing (HPC) resources and parallelized training frameworks. This connection highlights the suitability of SGL for supercomputing environments, especially in large-scale and real-time applications such as autonomous systems, medical imaging, and recommendation engines. Theoretical analysis further confirms the benefits of SGL in maintaining gradient stability, positioning it as a robust and HPC-relevant contribution to the field of deep learning optimization.
Keywords:
Smooth Gradient Loss
Deep Learning
Gradient Regularization
Loss Function
Machine Learning Optimization
Regularization Techniques
Journal
T
IF:
0
Papers:
647
Citations:
0
Organization
View more organizations