.png){kind=icon}
.png){kind=icon}
Return
Learning hierarchically-structured concepts
.png){kind=icon}
OA
AI
DOI：10.1016/j.neunet.2021.07.033.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We use a recently developed synchronous Spiking Neural Network (SNN) model to study the problem of learning hierarchically-structured concepts. We introduce an abstract data model that describes simple hierarchical concepts. We define a feed-forward layered SNN model, with learning modeled using Oja's local learning rule, a well known biologically-plausible rule for adjusting synapse weights. We define what it means for such a network to recognize hierarchical concepts; our notion of recognition is robust, in that it tolerates a bounded amount of noise. Then, we present a learning algorithm by which a layered network may learn to recognize hierarchical concepts according to our robust definition. We analyze correctness and performance rigorously; the amount of time required to learn each concept, after learning all of the sub-concepts, is approximately O (1/eta k (l(max) log(k) + 1/epsilon) + b log(k)), where k is the number of sub-concepts per concept, l(max) is the maximum hierarchical depth, eta is the learning rate, epsilon describes the amount of uncertainty allowed in robust recognition, and b describes the amount of weight decrease for irrelevant'' edges. An interesting feature of this algorithm is that it allows the network to learn sub-concepts in a highly interleaved manner. This algorithm assumes that the concepts are presented in a noise-free way; we also extend these results to accommodate noise in the learning process. Finally, we give a simple lower bound saying that, in order to recognize concepts with hierarchical depth two with noise-tolerance, a neural network should have at least two layers. The results in this paper represent first steps in the theoretical study of hierarchical concepts using SNNs. The cases studied here are basic, but they suggest many directions for extensions to more elaborate and realistic cases. (C) 2021 Elsevier Ltd. All rights reserved.
Keywords:
Hierarchical concepts
Representing hierarchical concepts
Recognizing hierarchical concepts
Learning hierarchical concepts
Spiking Neural Networks
Brain-inspired algorithms
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.