Optimizing Convolutions for Deep Learning Inference on ARM Cortex-M Processors

doi:10.1109/JIOT.2024.3395335

Optimizing Convolutions for Deep Learning Inference on ARM Cortex-M Processors

2024-08-01

1

OA

AI

DOI：10.1109/JIOT.2024.3395335

Original

Original request for help

Save

Abstract

En 中文

We perform a series of optimizations on the convolution operator within the ARM common microcontroller units software interface standard for neural network (CMSIS-NN) library to improve the performance of deep learning tasks on Arduino development boards equipped with ARM Cortex-M4 and M7 microcontrollers. To this end, we develop custom microkernels that efficiently handle the internal computations required by the convolution operator via the lowering approach and the direct method, and we design two techniques to avoid register spilling. We also take advantage of all the RAM on the Arduino boards by reusing it as a scratchpad for the convolution filters. The integration of these techniques into CMSIS-NN, when invoked by TensorFlow Lite for microcontrollers for quantised versions of VGG, SqueezeNet, ResNet, and MobileNet-like convolutional neural networks enhances the overall inference speed by a factor ranging from 1.13xto 1.50x .

Keywords:

Program processors

Random access memory

Registers

Optimization

Convolution

Inference algorithms

Signal processing algorithms

ARM Cortex-M

common microcontroller units software interface standard for neural network (CMSIS-NN)

convolution operator

deep learning

edge computing

high performance

microcontrollers

Optimizing Convolutions for Deep Learning Inference on ARM Cortex-M Processors

Abstract

Journal

Organization