FANE: FPGA-Based FP8 Approximate Neural Network Engine

Han Bao

doi:10.1109/TVLSI.2026.3677683

FANE: FPGA-Based FP8 Approximate Neural Network Engine

2026-05-04

0

PRE

AI

DOI：10.1109/TVLSI.2026.3677683

Original

Original request for help

Save

Abstract

En 中文

The 8-bit floating-point (FP8) format has gained growing interest in neural networks (NNs) for its superior dynamic range over traditional INT8. However, multiply-accumulate (MAC) operations remain a major source of power consumption during inference of NNs, which makes DSP-free design important, especially for edge FPGAs with few or no DSPs. Therefore, this brief presents FPGA-based FP8 approximate neural network engine (FANE), an FPGA-based approximate NN engine for FP8. We first introduce a novel approximation method that replaces the multiplications by linear additions. This approximate method reduces power consumption while maintaining high accuracy, outperforming the latest FP8 approximate multiplier by 53.15%. Based on this design, we construct an FP8 MAC unit and integrate it into both a convolution engine and a matrix–vector multiplication (MVM) unit. Finally, we integrate our design into a large language model (LLM). The result shows 61.5% higher efficiency (TOPS/W) than the previous design, demonstrating the superiority of FANE in terms of performance and power efficiency. The code of FANE is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://gitlab.com/etrovub/embedded-systems/publications/fane-fpga-based-fp8-approximate-neural-network-engine</uri>

Keywords:

8-bit floating-point (FP8)

approximate computing

FPGA

neural network (NN)

systolic array

FANE: FPGA-Based FP8 Approximate Neural Network Engine

Abstract

Journal

Organization