.png){kind=icon}
.png){kind=icon}
Return
A computationally efficient piecewise linear training algorithm for neural networks utilizing continuous special ordered sets
.png){kind=icon}
PRE
AI
DOI：10.1016/j.cherd.2026.03.047.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Artificial neural networks are commonly employed for data-driven modelling of complex nonlinear processes; however, their training may be hindered by the nonlinearity of activation functions and the dependence on local solvers. Obtaining an efficient global solution for neural network training continuous to be an unresolved challenge. A common strategy involves approximating activation functions using piecewise linear formulations to convexify the problem, however this often results in high computational costs due to the addition of auxiliary binary variables. This study proposes integrating piecewise linear formulations for neural network activation functions with a tailored branching algorithm that explores efficient linear programming relaxations to effectively explore the solution space. In the proposed framework, a subset of network parameters is obtained via regular, gradientdescent based training and kept fixed during the training process, while the remaining parameters are determined through the proposed formulation. Unlike conventional mixed-integer programming approaches, where the reliance on binary variables increases the computational complexity, the applied continuous special ordered set method achieves polynomial growth in computation, thereby ensuring better scalability for larger problem instances. The proposed method achieves minimal training error while significantly reducing CPU time. Experiments on datasets of equal dimensionality confirm that the efficiency of the algorithm is not dataset-specific, demonstrating consistent CPU time trends across multiple datasets. These findings highlight the generalizability of the suggested method and its potential to enhance artificial neural network training efficiency across various applications.
Keywords:
Artificial neural networks
Mixed integer linear programming
Piecewise linear functions
Special ordered set variables
Quasi
convex formulation
Journal
C
IF:
3.9
Papers:
83
Citations:
0