.png){kind=icon}
Return
Diluted binary neural network
.png){kind=icon}
PRE
AI
DOI：10.1016/j.patcog.2023.109556.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Binary neural networks (BNNs) are promising on resource-constrained devices because they reduce mem-ory consumption and accelerate inference effectively. However, they are still potential on performance improvement. Prior studies attribute performance degradation of BNNs to limited representation ability and gradient mismatch. In this paper, we find that it also results from the mandatory representation of small full-precision auxiliary weights to large values. To tackle with this issue, we propose an approach dubbed as Diluted Binary Neural Network (DBNN). Besides avoiding mandatory representation effectively, the proposed DBNN also alleviates sign flip problem to a large extent. For activations, we jointly min-imize quantization error and maximize information entropy to develop the binarization scheme. Com-pared with existing sparsity-binarization approaches, DBNN trains network from scratch without other procedures and achieves larger sparsity. Experiments on several datasets with various networks demon-strate the superiority of our approach. (c) 2023 Elsevier Ltd. All rights reserved.
Keywords:
Model compression
Network quantization
Binary neural network
Ternary neural network
Sparse regularization
Journal
IF:
7.6
Papers:
1.4W
Citations:
4.5W
Organization
Cited Papers
ShareSaveShareSaveNatural images are reliably represented by sparse and variable populations of neurons in visual cortex.png)
NATURE COMMUNICATIONS
OAAI
NATURE COMMUNICATIONSIF15.7
OAAIShareSaveShareSaveGXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization frameworkNEURAL NETWORKSOAAI
NEURAL NETWORKSIF6.3
OAAIShareSaveShareSave no more
no more