.png){kind=icon}
返回
Self-distribution binary neural networks
.png){kind=icon}
OA
AI
DOI：10.1007/s10489-022-03348-z.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this work, we study network binarization (i.e., binary neural networks, BNNs), which is one of the most promising techniques in network compression for convolutional neural networks (CNNs). Although prior work has introduced many binarization methods that improve the accuracy of BNNs by minimizing the quantization error, there remains a non-negligible performance gap between the binarized model and the full-precision model. Given that feature representation is critical for deep neural networks and that in BNNs, the features only differ in signs, we argue that the impact on the accuracy of BNNs may be strongly related to the sign distribution of the network parameters in addition to the quantization error. To this end, Self-Distribution Binary Neural Network (SD-BNN) is proposed. First, we utilize Activation Self Distribution (ASD) to adaptively adjust the sign distribution of activations, thereby improving the sign differences of the outputs of the convolution. Second, we adjust the sign distribution of weights through Weight Self Distribution (WSD) and then fine-tune the sign distribution of the outputs of the convolution. Extensive experiments on the CIFAR-10 and ImageNet datasets with various network structures show that the proposed SD-BNN consistently outperforms state-of-the-art (SOTA) BNNs (e.g., 92.5% on CIFAR-10 and 66.5% on ImageNet with ResNet-18) with lower computational cost. Our code is available at https://github.com/pingxue-hfut/SD-BNN.
Keyword:
BNNs
Self-distribution
Sign distribution
Feature representation
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
引用论文
A Hierarchical Cardiac Rhythm Classification Methodology Based on Electrocardiogram Fiducial Points一种基于心电图基准点的分层心律分类方法
OAAI
OAAI分享收藏An Efficient Binary Convolutional Neural Network With Numerous Skip Connections for Fog Computing
PREAI
分享收藏MXQN:Mixed quantization for reducing bit-width of weights and activations in deep convolutional neural networks.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
分享收藏STBNN: Hardware-friendly spatio-temporal binary neural network with high pattern recognition accuracySTBNN: 硬件友好的高模式识别精度时空二进制神经网络NEUROCOMPUTING
NEUROCOMPUTINGIF6.5
PREAI
分享收藏
查看更多内容