Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

Zhuang Shao

doi:10.1109/TCSI.2021.3120312

Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

2022-02-01

21

OA

AI

DOI：10.1109/TCSI.2021.3120312

Original

Original request for help

Save

Abstract

En 中文

Existing deep convolutional neural networks (CNNs) generate massive interlayer feature data during network inference. To maintain real-time processing in embedded systems, large on-chip memory is required to buffer the interlayer feature maps. In this paper, we propose an efficient hardware accelerator with an interlayer feature compression technique to significantly reduce the required on-chip memory size and off-chip memory access bandwidth. The accelerator compresses interlayer feature maps through transforming the stored data into frequency domain using hardware-implemented 8x 8 discrete cosine transform (DCT). The high-frequency components are removed after the DCT through quantization. Sparse matrix compression is utilized to further compress the interlayer feature maps. The on-chip memory allocation scheme is designed to support dynamic configuration of the feature map buffer size and scratch pad size according to different network-layer requirements. The hardware accelerator combines compression, decompression, and CNN acceleration into one computing stream, achieving minimal compressing and processing delay. A prototype accelerator is implemented on an FPGA platform and also synthesized in TSMC 28-nm COMS technology. It achieves 403GOPS peak throughput and 1.4x similar to 3.3x interlayer feature map reduction by adding light hardware area overhead, making it a promising hardware accelerator for intelligent IoT devices.

Keywords:

Deep convolution neural networks

discrete cosine transform

quantization

interlayer feature maps compression

Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

Abstract

Journal

Organization