.png){kind=icon}
返回
Training binary neural networks with knowledge transfer
.png){kind=icon}
OA
AI
DOI：10.1016/j.neucom.2018.09.103.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Binary Neural Networks (BNNs) use binary values for both weights and activations instead of 32 bit floating point numbers typically used in deep neural networks. This reduces the memory footprint by a factor of 32 and allows a very efficient implementation in hardware. BNNs are trained using regular gradient descent but are harder to optimise, take longer to train and generally require a more careful tuning of hyperparameters such as the learning rate decay schedule than floating point versions. We propose to use Knowledge Transfer techniques to make it easier to train BNNs. Knowledge transfer is a general technique that tries to transfer the knowledge stored in a large network (the teacher) to a smaller (student) network. In our case the teacher is a network trained with floating point weights and activations while the student is a BNN. We apply different Knowledge Transfer techniques to the task of training a BNN. We introduce a novel similarity based Knowledge Transfer algorithm and show that this technique results in a higher test accuracy on different benchmark datasets compared to training the BNN from scratch. (C) 2019 Elsevier B.V. All rights reserved.
Keyword:
Binary neural networks
Deep learning
Knowledge transfer
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.5
论文数:
2.5W
被引数:
6.5W
机构
引用论文
Stabilization of desired periodic orbits in dynamic binary neural networks动态二元神经网络中期望周期轨道的稳定化.png)
NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏分享收藏Point-of-care ultrasonography adoption in Canada: using diffusion theory and the Evaluation Tool for Ultrasound skills Development and Education (ETUDE)CJEMOAAI
CJEMIF0
OAAI分享收藏When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs当深度学习满足度量学习: 通过学习判别cnn进行遥感图像场景分类
PREAI
分享收藏 没有更多内容
没有更多内容