An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Xiaolei Zhu

doi:10.1109/TCDS.2021.3073846

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

2022-09-01

13

PRE

AI

DOI：10.1109/TCDS.2021.3073846

Original

Original request for help

Save

Abstract

En 中文

It is challenging to train deep spiking neural networks (SNNs) directly due to the difficulties associated with the nondifferentiable neuron model. In this work, an end-to-end learning algorithm based on discrete current-based leaky integrate-and-fire (C-LIF) neuron model and surrogate gradient is proposed to leverage the encoder-decoder network architecture to train deep SNNs directly. The proposed algorithm is capable of learning deep spatiotemporal features relying on current time step only, and several acceleration techniques including backward phase skipping and layerwise FreezeOut are proposed to accelerate the training. Experimental results show that the proposed learning algorithm achieved the classification accuracies of 98.40% and 95.83% on the dynamic neuromorphic data sets MNIST-DVS and DVS-Gestures, respectively, and of 99.58% and 95.97% on static vision data sets MNIST and SVHN, respectively, which are comparable to the existing state-of-the-art results. The training speed was accelerated by up to 49.5% on MNIST and 36.6% on DVS-Gestures with the proposed acceleration techniques while maintaining the same level of accuracy.

Keywords:

Neurons

Training

Biological system modeling

Acceleration

Heuristic algorithms

Backpropagation

Biological neural networks

Backward phase skipping

layerwise FreezeOut (LFO)

leaky integrate-and-fire (LIF) neuron model

spiking neural networks (SNNs)

surrogate gradient

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Abstract

AI Summary

Journal

Organization