SAMBA: Sparsity Aware In-Memory Computing Based Machine Learning Accelerator

doi:10.1109/TC.2023.3257513

SAMBA: Sparsity Aware In-Memory Computing Based Machine Learning Accelerator

2023-09-01

3

PRE

AI

DOI：10.1109/TC.2023.3257513

Original

Original request for help

Save

Abstract

En 中文

Machine Learning (ML) inference is typically dominated by highly data-intensive Matrix Vector Multiplication (MVM) computations that may be constrained by memory bottleneck due to massive data movement between processor and memory. Although analog in-memory computing (IMC) ML accelerators have been proposed to execute MVM with high efficiency, the latency and energy of such computing systems can be dominated by the large latency and energy costs from analog-to-digital converters (ADCs). Leveraging sparsity in ML workloads, reconfigurable ADCs can save MVM energy and latency by reducing the required ADC bit precision. However, such improvement in latency can be hindered by non-uniform sparsity of the weight matrices mapped into hardware. Moreover, data movement between MVM processing cores may become another factor that delays the overall system-level performance. To address these issues, we propose SAMBA, Sparsity Aware IMC Based Machine Learning Accelerator. First, we propose load balancing during mapping of weight matrices into physical crossbars to eliminate non-uniformity in the sparsity of mapped matrices. Second, we propose optimizations in arranging and scheduling the tiled MVM hardware to minimize the overhead of data movement across multiple processing cores. Our evaluations show that the proposed load balancing technique can achieve performance improvement. The proposed optimizations can further improve both performance and energy-efficiency regardless of sparsity condition. With the combination of load balancing and data movement optimization in conjunction with reconfigurable ADCs, our proposed approach provides up to 2.38x speed-up and 1.54x energy-efficiency over stateof- art analog IMC based ML accelerators for ImageNet datasets on Resnet-50 architecture.

Keywords:

Computer architecture

Optimization

Load management

Convolution

In-memory computing

Hardware

Energy consumption

Accelerator

in-memory computing

neural networks

sparsity

SAMBA: Sparsity Aware In-Memory Computing Based Machine Learning Accelerator

Abstract

Journal

Organization