.png){kind=icon}
返回
SNIB: Improving Spike-Based Machine Learning Using Nonlinear Information Bottleneck
.png){kind=icon}
PRE
AI
DOI：10.1109/TSMC.2023.3300318.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Spiking neural networks (SNNs) have garnered increased attention in the field of artificial general intelligence (AGI) research due to their low power consumption, high computational efficiency, and low latency induced by their event-driven and sparse communication features. However, efficiently and robustly training an SNN presents a challenge. In this study, we introduce a novel framework for spike-based machine learning called spike-based nonlinear information bottleneck (SNIB). This framework utilizes an information-theoretic learning (ITL) approach and a surrogate gradient learning (SGL) method to achieve robust, accurate, and low-power performance. The proposed SNIB framework includes three variants: 1) squared information bottleneck (SIB); 2) cubic information bottleneck (CIB); and 3) quartic information bottleneck (QIB) strategies, which use a mapping mechanism to compress spiking representations. We systematically evaluate these strategies using different types of input noise and neuromorphic hardware noise. Our experimental results demonstrate that all three strategies effectively enhance the robustness of SGL in SNN architectures. Furthermore, SNIB can significantly reduce the power consumption of SNNs. As a result, SNIB offers a new and significant perspective for hardware-constrained general mobile devices for embedded edge intelligence and represents a progressive step toward realizing AGI.
Keyword:
Brain-inspired intelligence
information bottleneck (IB)
information-theoretic learning (ITL)
neuromorphic computing
spiking neural network (SNN)
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
Organized Resting-state Functional Dysconnectivity of the Prefrontal Cortex in Patients with Schizophrenia.png)
Neuroscience
NeuroscienceIF0
PREAI
分享收藏
Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks大规模基于电导的脉冲神经网络的实时神经形态系统
PREAI
分享收藏Contributions of evacuation proctography and anorectal manometry to evaluation of adults with constipation and defecatory difficulty排空直肠造影和肛门直肠测压对成人便秘和排便困难的评估的贡献
PREAI
分享收藏分享收藏
查看更多内容