.png){kind=icon}
Return
SNIB: Improving Spike-Based Machine Learning Using Nonlinear Information Bottleneck
.png){kind=icon}
PRE
AI
DOI：10.1109/TSMC.2023.3300318.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Spiking neural networks (SNNs) have garnered increased attention in the field of artificial general intelligence (AGI) research due to their low power consumption, high computational efficiency, and low latency induced by their event-driven and sparse communication features. However, efficiently and robustly training an SNN presents a challenge. In this study, we introduce a novel framework for spike-based machine learning called spike-based nonlinear information bottleneck (SNIB). This framework utilizes an information-theoretic learning (ITL) approach and a surrogate gradient learning (SGL) method to achieve robust, accurate, and low-power performance. The proposed SNIB framework includes three variants: 1) squared information bottleneck (SIB); 2) cubic information bottleneck (CIB); and 3) quartic information bottleneck (QIB) strategies, which use a mapping mechanism to compress spiking representations. We systematically evaluate these strategies using different types of input noise and neuromorphic hardware noise. Our experimental results demonstrate that all three strategies effectively enhance the robustness of SGL in SNN architectures. Furthermore, SNIB can significantly reduce the power consumption of SNNs. As a result, SNIB offers a new and significant perspective for hardware-constrained general mobile devices for embedded edge intelligence and represents a progressive step toward realizing AGI.
Keywords:
Brain-inspired intelligence
information bottleneck (IB)
information-theoretic learning (ITL)
neuromorphic computing
spiking neural network (SNN)
Journal
IF:
10.5
Papers:
1.1W
Citations:
5.0W
Organization
Cited Papers
ShareSaveOrganized Resting-state Functional Dysconnectivity of the Prefrontal Cortex in Patients with Schizophrenia.png)
Neuroscience
NeuroscienceIF0
PREAI
ShareSave
ShareSaveShareSaveShareSave
View more