.png){kind=icon}
.png){kind=icon}
Return
Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TCSII.2020.3002564.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The softmax function has been widely used in deep neural networks (DNNs), and studies on efficient hardware accelerators for DNN have also attracted tremendous attention. However, it is very challenging to design efficient hardware architectures for softmax because of the expensive exponentiation and division calculations in it. In this brief, the softmax function is firstly simplified by exploring algorithmic strength reductions. Afterwards, a hardware-friendly and precision-adjustable calculation method for softmax is proposed, which can meet different precision requirements in various deep learning (DL) tasks. Based on the above innovations, an efficient architecture for softmax is presented. By tuning the parameter P, the system accuracy and complexity can be adjusted dynamically to achieve a good tradeoff between them. The proposed design is coded using hardware description language (HDL) and evaluated on two platforms, Xilinx Zynq-7000 ZC706 development board and TSMC 28-nm CMOS technology, respectively. Hardware implementation results show that our architecture significantly outperforms other works in speed and area, and that by adjusting P, the accuracy can be further increased with little hardware overhead.
Keywords:
Hardware
Computer architecture
Complexity theory
Neural networks
Circuits and systems
Task analysis
Mathematical model
Softmax
hardware architecture
deep neural network
transformer
VLSI
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
I
IF:
4.9
Papers:
8.8K
Citations:
2.5W