HIMAM: Hardware Implementation of Multiply-and-Max/Min Layers for Energy-Efficient DNN Inference

Fanny Spagnolo

doi:10.1109/TCSII.2025.3581784

HIMAM: Hardware Implementation of Multiply-and-Max/Min Layers for Energy-Efficient DNN Inference

2025-08-01

0

PRE

AI

F

Fanny Spagnolo

P

Pasquale Corsonello

S

Stefania Perri

DOI：10.1109/TCSII.2025.3581784

Original

Original request for help

Save

Abstract

En 中文

This Brief presents HIMAM: the first hardware implementation of the Multiply-and-Max/Min (MAM) layers, recently proposed as an effective alternative to the traditional Multiply-and-Accumulate (MAC) paradigm used in Deep Neural Networks (DNNs). The proposed design relies on a specialized hardware architecture that uses floating-point arithmetic and was devised to implement the unconventional multiply then compare-and-add pipeline involved in MAM layers. Based on the observation that such a paradigm actually requires just few product operations to be accurately computed, we propose to replace the most computational intensive components with approximate ones. The FPGA-based implementation carried out on a Zynq Ultrascale+ device and operating in 32-bit floating-point mode exhibits 10.91 GFLOPS/W. When implemented on a 28-nm FDSOI technology process, such an architecture dissipates only 5.3 mW running at 250 MHz, which is at least 41.7% lower than the MAC-based state-of-the-art hardware architectures.

Keywords:

Deep neural networks

Multiply-and-Max/Min layers

energy-efficient design

hardware accelerators

approximate computing

HIMAM: Hardware Implementation of Multiply-and-Max/Min Layers for Energy-Efficient DNN Inference

Abstract

Journal

Organization

Cited Papers

Citing Papers