Toward Deep Adaptive Hinging Hyperplanes

doi:10.1109/TNNLS.2021.3079113

Toward Deep Adaptive Hinging Hyperplanes

2022-11-01

5

OA

AI

DOI：10.1109/TNNLS.2021.3079113

原文链接

查看原文

摘要

En 中文

The adaptive hinging hyperplane (AHH) model is a popular piecewise linear representation with a generalized tree structure and has been successfully applied in dynamic system identification. In this article, we aim to construct the deep AHH (DAHH) model to extend and generalize the networking of AHH model for high-dimensional problems. The network structure of DAHH is determined through a forward growth, in which the activity ratio is introduced to select effective neurons and no connecting weights are involved between the layers. Then, all neurons in the DAHH network can be flexibly connected to the output in a skip-layer format, and only the corresponding weights are the parameters to optimize. With such a network framework, the backpropagation algorithm can be implemented in DAHH to efficiently tackle large-scale problems and the gradient vanishing problem is not encountered in the training of DAHH. In fact, the optimization problem of DAHH can maintain convexity with convex loss in the output layer, which brings natural advantages in optimization. Different from the existing neural networks, DAHH is easier to interpret, where neurons are connected sparsely and analysis of variance (ANOVA) decomposition can be applied, facilitating to revealing the interactions between variables. A theoretical analysis toward universal approximation ability and explicit domain partitions are also derived. Numerical experiments verify the effectiveness of the proposed DAHH.

Keyword:

Neurons

Artificial neural networks

Network topology

Training

Topology

Optimization

Adaptive systems

Adaptive hinging hyperplanes (AHHs)

analysis of variance (ANOVA) decomposition

domain partition

piecewise linear (PWL)

skip-layer connection

Toward Deep Adaptive Hinging Hyperplanes

摘要

AI总结

期刊

机构