.png){kind=icon}
Return
Block-term tensor neural networks
.png){kind=icon}
OA
AI
DOI：10.1016/j.neunet.2020.05.034.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Deep neural networks (DNNs) have achieved outstanding performance in a wide range of applications, e.g., image classification, natural language processing, etc. Despite the good performance, the huge number of parameters in DNNs brings challenges to efficient training of DNNs and also their deployment in low-end devices with limited computing resources. In this paper, we explore the correlations in the weight matrices, and approximate the weight matrices with the low-rank block term tensors. We name the new corresponding structure as block-term tensor layers (BT-layers), which can be easily adapted to neural network models, such as CNNs and RNNs. In particular, the inputs and the outputs in BT-layers are reshaped into low-dimensional high-order tensors with a similar or improved representation power. Sufficient experiments have demonstrated that BT-layers in CNNs and RNNs can achieve a very large compression ratio on the number of parameters while preserving or improving the representation power of the original DNNs. (C) 2020 Elsevier Ltd. All rights reserved.
Keywords:
Tensor networks
Network compression
Neural networks
Deep learning
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.3
Papers:
8.2K
Citations:
3.0W
Organization
View more organizations
Cited Papers
ShareSaveContributions of Valine-292 in the Nicotinamide Binding Site of Liver Alcohol Dehydrogenase and Dynamics to Catalysis,.png)
Biochemistry
BiochemistryIF0
PREAI
ShareSaveShareSaveAccelerating deep neural network training with inconsistent stochastic gradient descentNEURAL NETWORKS
NEURAL NETWORKSIF6.3
PREAI
ShareSaveShareSaveShareSave View more