A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid Function

Zhe Pan

doi:10.1109/TIE.2022.3146573

A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid Function

2022-10-01

20

PRE

AI

DOI：10.1109/TIE.2022.3146573

Original

Original request for help

Save

Abstract

En 中文

The sigmoid function is a widely used nonlinear activation function in neural networks. In this article, we present a modular approximation methodology for efficient fixed-point hardware implementation of the sigmoid function. Our design consists of three modules: piecewise linear (PWL) approximation as the initial solution, Taylor series approximation of the exponential function, and Newton-Raphson method-based approximation as the final solution. Its modularity enables the designer to flexibly choose the most appropriate approximation method for each module separately. Performance evaluation results indicate that our work strikes an appropriate balance among the objectives of approximation accuracy, hardware resource utilization, and performance.

Keywords:

Hardware

Taylor series

Newton method

Approximation error

Artificial neural networks

Quantization (signal)

Input variables

Artificial neural networks (NNs)

FPGA

hardware acceleration

Newton-Raphson (NR) method

sigmoid function

A Modular Approximation Methodology for Efficient Fixed-Point Hardware Implementation of the Sigmoid Function

Abstract

Journal

Organization