.png){kind=icon}
返回
Supporting the Momentum Training Algorithm Using a Memristor-Based Synapse
.png){kind=icon}
OA
AI
DOI：10.1109/TCSI.2018.2888538.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Despite the increasing popularity of deep neural networks (DNNs), they cannot he trained efficiently on existing platforms, and efforts have thus been devoted to designing dedicated hardware for DNNs. In our recent work, we have provided direct support for the stochastic gradient descent (SGD) training algorithm by constructing the basic element of neural networks, the synapse, using emerging technologies, namely memristors. Due to the limited performance of SGD, optimization algorithms are commonly employed in DNN training. Therefore, DNN accelerators that only support SGD might not meet DNN training requirements. In this paper, we present a memristor-based synapse that supports the commonly used momentum algorithm. Momentum significantly improves the convergence of SGD and facilitates the DNN training stage. We propose two design approaches to support momentum: 1) a hardware friendly modification of the momentum algorithm using memory external to the synapse structure, and 2) updating each synapse with a built-in memory. Our simulations show that the proposed DNN training solutions are as accurate as training on a GPU platform while speeding up the performance by 886 x and decreasing energy consumption by 7 x, on average.
Keyword:
Memristor
deep neural networks
training
momentum
synapse
stochastic gradient descent
hardware
VTEAM
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
5.2
论文数:
9.8K
被引数:
2.2W
机构
引用论文
Designing a neural network for forecasting financial and economic time series.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source以五氧化二磷为掺杂源合成n型半导体金刚石膜
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
查看更多内容