Neural network training as a dissipative process

Marco Gori

doi:10.1016/j.neunet.2016.05.005

2016-09-01

PRE

Marco Gori

Marco Maggini

Alessandro Rossi *

DOI：10.1016/j.neunet.2016.05.005

原文链接

摘要

En 中文

This paper analyzes the practical issues and reports some results on a theory in which learning is modeled as a continuous temporal process driven by laws describing the interactions of intelligent agents with their own environment. The classic regularization framework is paired with the idea of temporal manifolds by introducing the principle of least cognitive action, which is inspired by the related principle of mechanics. The introduction of the counterparts of the kinetic and potential energy leads to an interpretation of learning as a dissipative process. As an example, we apply the theory to supervised learning in neural networks and show that the corresponding Euler Lagrange differential equations can be connected to the classic gradient descent algorithm on the supervised pairs. We give preliminary experiments to confirm the soundness of the theory. (C) 2016 Elsevier Ltd. All rights reserved.

Keyword:

Temporal manifolds

Regularization networks

Dissipative systems

On-line back-propagation