.png){kind=icon}
返回
Enhanced regularization for on-chip training using analog and temporary memory weights
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2023.07.001.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In-memory computing techniques are used to accelerate artificial neural network (ANN) training and inference tasks. Memory technology and architectural innovations allow efficient matrix-vector multiplications, gradient calculations, and updates to network weights. However, on-chip learning for edge devices is quite challenging due to the frequent updates. Here, we propose using an analog and temporary on-chip memory (ATOM) cell with controllable retention timescales for implementing the weights of an on-chip training task. Measurement results for Read-Write timescales are presented for an ATOM cell fabricated in GlobalFoundries' 45 nm RFSOI technology. The effect of limited retention and its variability is evaluated for training a fully connected neural network with a variable number of layers for the MNIST hand-written digit recognition task. Our studies show that weight decay due to temporary memory can have benefits equivalent to regularization, achieving a -33% reduction in the validation error (from 3.6% to 2.4%). We also show that the controllability of the decay timescale can be advantageous in achieving a further -26% reduction in the validation error. This strongly suggests the utility of temporary memory during learning before on-chip non-volatile memories can take over for the storage and inference tasks using the neural network weights. We thus propose an algorithm -circuit codesign in the form of temporary analog memory for high-performing on-chip learning of ANNs.(c) 2023 Elsevier Ltd. All rights reserved.
Keyword:
Temporary memory
Regularization
On-chip learning
Artificial neural network
In-memory computing
ML hardware
期刊
IF:
6.3
论文数:
8.2K
被引数:
3.0W
机构
引用论文
Determination of genetic changes of Rev-erb beta and Rev-erb alpha genes in Type 2 diabetes mellitus by next-generation sequencing.png)
Gene
GeneIF0
PREAI
分享收藏An adaptive synaptic array using Fowler-Nordheim dynamic analog memoryNATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏The Next Generation of Deep Learning Hardware: Analog Computing下一代深度学习硬件: 模拟计算PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏Analog architectures for neural network acceleration based on non-volatile memory基于非易失性存储器的神经网络加速模拟架构APPLIED PHYSICS REVIEWSOAAI
APPLIED PHYSICS REVIEWSIF11.6
OAAI分享收藏Strength investigation of a small size floating dock unit by 3D-FEM models in head design waves通过头部设计波中的3D-FEM模型对小型浮船坞单元进行强度研究OAAI
OAAI分享收藏
查看更多内容