Desynchronous learning in a physics-driven learning network

doi:10.1063/5.0084631

Desynchronous learning in a physics-driven learning network

2022-04-11

18

OA

AI

DOI：10.1063/5.0084631

Original

Original request for help

Save

Abstract

En 中文

In a neuron network, synapses update individually using local information, allowing for entirely decentralized learning. In contrast, elements in an artificial neural network are typically updated simultaneously using a central processor. Here, we investigate the feasibility and effect of desynchronous learning in a recently introduced decentralized, physics-driven learning network. We show that desynchronizing the learning process does not degrade the performance for a variety of tasks in an idealized simulation. In experiment, desynchronization actually improves the performance by allowing the system to better explore the discretized state space of solutions. We draw an analogy between desynchronization and mini-batching in stochastic gradient descent and show that they have similar effects on the learning process. Desynchronizing the learning process establishes physics-driven learning networks as truly fully distributed learning machines, promoting better performance and scalability in deployment.

Keywords:

NEURAL-NETWORKS

Desynchronous learning in a physics-driven learning network

Abstract

Journal

Organization