.png){kind=icon}
返回
Memory-Efficient Deep Learning on a SpiNNaker 2 Prototype
.png){kind=icon}
OA
AI
DOI：10.3389/fnins.2018.00840.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The memory requirement of deep learning algorithms is considered incompatible with the memory restriction of energy-efficient hardware. A low memory footprint can be achieved by pruning obsolete connections or reducing the precision of connection strengths after the network has been trained. Yet, these techniques are not applicable to the case when neural networks have to be trained directly on hardware due to the hard memory constraints. Deep Rewiring (DEEP R) is a training algorithm which continuously rewires the network while preserving very sparse connectivity all along the training procedure. We apply DEEP R to a deep neural network implementation on a prototype chip of the 2nd generation SpiNNaker system. The local memory of a single core on this chip is limited to 64 KB and a deep network architecture is trained entirely within this constraint without the use of external memory. Throughout training, the proportion of active connections is limited to 1.3%. On the handwritten digits dataset MNIST, this extremely sparse network achieves 96.6% classification accuracy at convergence. Utilizing the multi-processor feature of the SpiNNaker system, we found very good scaling in terms of computation time, per-core memory consumption, and energy constraints. When compared to a X86 CPU implementation, neural network training on the SpiNNaker 2 prototype improves power and energy consumption by two orders of magnitude.
Keyword:
deep rewiring
pruning
sparsity
SpiNNaker
memory footprint
parallelism
energy efficient hardware
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.2
论文数:
1.6W
被引数:
5.3W
机构
引用论文
.png){kind=icon}
Contributions of evacuation proctography and anorectal manometry to evaluation of adults with constipation and defecatory difficulty排空直肠造影和肛门直肠测压对成人便秘和排便困难的评估的贡献
PREAI
分享收藏分享收藏
查看更多内容