A Novel Approximation Methodology and Its Efficient VLSI Implementation for the Sigmoid Function

doi:10.1109/TCSII.2020.2999458

A Novel Approximation Methodology and Its Efficient VLSI Implementation for the Sigmoid Function

2020-12-01

23

PRE

AI

DOI：10.1109/TCSII.2020.2999458

Original

Original request for help

Save

Abstract

En 中文

In this brief, a novel approximation method and its optimized hardware implementation are proposed for the sigmoid function used in Deep Neural Networks (DNNs). Based on piecewise approximation and truncated Taylor series expansion, the proposed method achieves very good approximation with low complexity while exploiting data representation with powers of two. In addition, by analyzing gradients of the sigmoid function, a small trick is introduced to improve the approximation precision. Furthermore, to reduce the hardware complexity and shorten the critical path, sampled values of the function are generated with simple logical-mapping. It is shown that the proposed approximation schemes can be implemented with purely combinational logic and the sigmoid function can be computed in one clock cycle. The experimental results demonstrate that the mean absolute errors are at the order of 1 x 10(-3). Compared with prior arts, the new design can obtain significant improvement in critical path with comparable performance.

Keywords:

Hardware

Approximation methods

Complexity theory

Table lookup

Mathematical model

Circuits and systems

Sigmoid function

approximation method

VLSI architecture

A Novel Approximation Methodology and Its Efficient VLSI Implementation for the Sigmoid Function

Abstract

Journal

Organization