.png){kind=icon}
返回
Automatic Learning Rate Adaption for Memristive Deep Learning Systems
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2023.3244006.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
As a possible device to further enhance the performance of the hybrid complementary metal oxide semiconductor (CMOS) technology in the hardware, the memristor has attracted widespread attention in implementing efficient and compact deep learning (DL) systems. In this study, an automatic learning rate tuning method for memristive DL systems is presented. Memristive devices are utilized to adjust the adaptive learning rate in deep neural networks (DNNs). The speed of the learning rate adaptation process is fast at first and then becomes slow, which consist of the memristance or conductance adjustment process of the memristors. As a result, no manual tuning of learning rates is required in the adaptive back propagation (BP) algorithm. While cycle-to-cycle and device-to-device variations could be a significant issue in memristive DL systems, the proposed method appears robust to noisy gradients, various architectures, and different datasets. Moreover, fuzzy control methods for adaptive learning are presented for pattern recognition, such that the over-fitting issue can be well addressed. To our best knowledge, this is the first memristive DL system using an adaptive learning rate for image recognition. Another highlight of the presented memristive adaptive DL system is that quantized neural network architecture is utilized, and there is therefore a significant increase in the training efficiency, without the loss of testing accuracy.
Keyword:
Memristors
Adaptive learning
Neural networks
Adaptation models
Image recognition
Computer architecture
Tuning
Adaptive learning rate
deep learning (DL)
fuzzy logic
image recognition
memristor
quantized neural network
期刊
IF:
8.9
论文数:
7.6K
被引数:
7.2W
机构
引用论文
Activity-Dependent Synaptic Plasticity of a Chalcogenide Electronic Synapse for Neuromorphic Systems神经形态系统的硫族化物电子突触的活动依赖性突触可塑性.png)
SCIENTIFIC REPORTS
OAAI
SCIENTIFIC REPORTSIF3.9
OAAI分享收藏Efficient and self-adaptive in-situ learning in multilayer memristor neural networks多层忆阻器神经网络的高效自适应原位学习NATURE COMMUNICATIONSOAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏An Ultra-Low Power Binarized Convolutional Neural Network-Based Speech Recognition Processor With On-Chip Self-Learning基于片上自学习的超低功耗二值化卷积神经网络语音识别处理器
PREAI
分享收藏
查看更多内容