.png){kind=icon}
返回
Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization
.png){kind=icon}
OA
AI
DOI：10.1109/TNNLS.2021.3109064.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
As well known, the huge memory and compute costs of both artificial neural networks (ANNs) and spiking neural networks (SNNs) greatly hinder their deployment on edge devices with high efficiency. Model compression has been proposed as a promising technique to improve the running efficiency via parameter and operation reduction, whereas this technique is mainly practiced in ANNs rather than SNNs. It is interesting to answer how much an SNN model can be compressed without compromising its functionality, where two challenges should be addressed: 1) the accuracy of SNNs is usually sensitive to model compression, which requires an accurate compression methodology and 2) the computation of SNNs is event-driven rather than static, which produces an extra compression dimension on dynamic spikes. To this end, we realize a comprehensive SNN compression through three steps. First, we formulate the connection pruning and weight quantization as a constrained optimization problem. Second, we combine spatiotemporal backpropagation (STBP) and alternating direction method of multipliers (ADMMs) to solve the problem with minimum accuracy loss. Third, we further propose activity regularization to reduce the spike events for fewer active operations. These methods can be applied in either a single way for moderate compression or a joint way for aggressive compression. We define several quantitative metrics to evaluate the compression performance for SNNs. Our methodology is validated in pattern recognition tasks over MNIST, N-MNIST, CIFAR10, and CIFAR100 datasets, where extensive comparisons, analyses, and insights are provided. To the best of our knowledge, this is the first work that studies SNN compression in a comprehensive manner by exploiting all compressible components and achieves better results.
Keyword:
Neurons
Computational modeling
Quantization (signal)
Optimization
Encoding
Task analysis
Synapses
Activity regularization
alternating direction method of multiplier (ADMM)
connection pruning
spiking neural network (SNN) compression
weight quantization
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
The Human Immunodeficiency Virus Type 1 Vpu Protein Enhances Membrane Permeability.png)
Biochemistry
BiochemistryIF0
PREAI
分享收藏Contributions of evacuation proctography and anorectal manometry to evaluation of adults with constipation and defecatory difficulty排空直肠造影和肛门直肠测压对成人便秘和排便困难的评估的贡献
PREAI
分享收藏分享收藏A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications在内存计算应用程序的尖峰神经网络训练过程中应用的软修剪方法
OAAI
OAAI分享收藏
查看更多内容