.png){kind=icon}
返回
A Fully Probabilistic Model for Sigmoid Approximation and Its Hardware-Efficient Implementation
.png){kind=icon}
PRE
AI
DOI：10.1109/TCSI.2024.3413570.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The sigmoid function is a representative activation function in shallow neural networks. Its hardware realization is challenging due to the complex exponential and reciprocal operations. Existing studies applied piecewise models to approximate sigmoid function and employed numerical methods or non-uniform input segmentations to mitigate fitting inaccuracies. However, the breakpoints introduce inevitable approximation precision loss. Besides, additional fitting processes greatly increase hardware complexity and power consumption. This paper presents a hardware-friendly sigmoidal approximation from the perspective of probability theory. We find that for a given input, the output of a sigmoid function can be approximated by the probability that the sum of this input and a Gaussian random variable is greater than or equal to zero. As the derived theorem does not involve piecewise expressions, the precision loss caused by the breakpoint issue is avoided. A low-complexity binary-search-based address localization method is proposed to optimize our theorem for hardware implementation. For the optimized scheme, an efficient implemented circuit is also presented. Our scheme's approximation ability and hardware efficiency are validated through software modeling and FPGA-and ASIC-based experiments. Feedforward neural network-based classification applications demonstrate that building networks with the proposed sigmoid approximator has only a tiny recognition rate loss.
Keyword:
Hardware
Probabilistic logic
Numerical models
Computational modeling
Random variables
Optimization
Integrated circuit modeling
FPGA implementation
ASIC implementation
hardware efficiency
probabilistic-based method
sigmoid activation function
期刊
IF:
5.2
论文数:
9.8K
被引数:
2.2W
机构
引用论文
.png){kind=icon}
Piezoresistive Silicon Microresonator for Measurements of Hydrogen Adsorption in Carbon Nanotubes
PREAI
分享收藏A Shallow Neural Network for Real-Time Embedded Machine Learning for Tensorial Tactile Data Processing用于实时嵌入式机器学习的浅层神经网络，用于张量触觉数据处理
OAAI
OAAI分享收藏
A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid FunctionSigmoid函数的高效定点硬件实现的模块化近似方法
PREAI
分享收藏
查看更多内容