.png){kind=icon}
返回
Diluted binary neural network
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109556.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Binary neural networks (BNNs) are promising on resource-constrained devices because they reduce mem-ory consumption and accelerate inference effectively. However, they are still potential on performance improvement. Prior studies attribute performance degradation of BNNs to limited representation ability and gradient mismatch. In this paper, we find that it also results from the mandatory representation of small full-precision auxiliary weights to large values. To tackle with this issue, we propose an approach dubbed as Diluted Binary Neural Network (DBNN). Besides avoiding mandatory representation effectively, the proposed DBNN also alleviates sign flip problem to a large extent. For activations, we jointly min-imize quantization error and maximize information entropy to develop the binarization scheme. Com-pared with existing sparsity-binarization approaches, DBNN trains network from scratch without other procedures and achieves larger sparsity. Experiments on several datasets with various networks demon-strate the superiority of our approach. (c) 2023 Elsevier Ltd. All rights reserved.
Keyword:
Model compression
Network quantization
Binary neural network
Ternary neural network
Sparse regularization
期刊
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
分享收藏分享收藏Graphene/Ionic Liquid Binary Electrode Material for High Performance Supercapacitor用于高性能超级电容器的石墨烯/离子液体二元电极材料
PREAI
分享收藏Natural images are reliably represented by sparse and variable populations of neurons in visual cortex.png)
NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAI分享收藏Energy-efficient load balancing scheme for two-tier communication in wireless sensor networks无线传感器网络中节能的两层通信负载均衡方案
PREAI
分享收藏GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization frameworkNEURAL NETWORKSOAAI
NEURAL NETWORKSIF6.3
OAAI分享收藏分享收藏 没有更多内容
没有更多内容