Event stream learning using spatio-temporal event surface

doi:10.1016/j.neunet.2022.07.010

Event stream learning using spatio-temporal event surface

2022-10-01

8

PRE

AI

DOI：10.1016/j.neunet.2022.07.010

Original

Original request for help

Save

Abstract

En 中文

Event cameras sense changes in light intensity and record them as an asynchronous event stream. Efficiently encoding and learning spatiotemporal information of the event streams remain challenging. In this paper, we propose a novel event descriptor to encode the spatio-temporal features for event streams and a local-search based multi-spike learning algorithm for spiking neural networks to classify encoded features. The spatio-temporal event surface (STES) descriptor explicitly captures both spatial and temporal correlations among events, and thus can characterize spatiotemporal features more accurately than existing feature descriptors that focus only on temporal or spatial information. In classification with multi-spike learning, we introduce a local search and gradient clipping mechanism to ensure the efficiency and stability of learning, which avoids other multi-spike learning rules' time-consuming global search and the gradient explosion problem. Experimental results demonstrate the superior classification performance of our proposed model, especially for event streams with rich spatiotemporal dynamics. (C) 2022 Elsevier Ltd. All rights reserved.

Keywords:

Spatiotemporal feature descriptor

Spike -based learning

Event streams classification

Spiking neural network

Event stream learning using spatio-temporal event surface

Abstract

Journal

Organization