.png){kind=icon}
.png){kind=icon}
Return
Batched data layout optimization for Im2col-based convolutions on CPUs
.png){kind=icon}
PRE
AI
DOI：10.1007/s11227-025-07706-9.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Im2col-based convolution is a widely used technique in deep learning frameworks to efficiently implement convolution operators. However, in batched input scenarios, it suffers from inefficient weight reuse due to large reuse distances, often becoming a performance bottleneck. Particularly for CPUs with cache, large reuse distances of convolution weight data lead to increased memory access latency and degraded performance. To address this issue, we propose a novel batched-data-layout-optimization (BDLO) method. BDLO optimizes the input matrix layouts during the Im2col phase to reduce reuse distances of weight data, thereby enhancing memory access efficiency in batched Im2col-based convolutions. This method improves performance of batched convolutions on CPUs and is available for both NCHW and NHWC data layouts. Experimental results demonstrate that BDLO outperforms the standard batched Im2col-based convolutions implemented in the widely used open-source deep learning framework LibTorch, while the weight matrix size is larger than the input matrix size. Specifically, BDLO achieves up to 1.40 $$\times$$ speedup for the NCHW layout and 1.37 $$\times$$ for the NHWC layout on the Kunpeng 920 CPU (ARM architecture), and up to 1.81 $$\times$$ for NCHW and 2.33 $$\times$$ for NHWC on the Intel Xeon Platinum 8260L CPU (x86 architecture).
Keywords:
Im2col-based convolution
Batched convolutions
CPUs
Reuse distance
Data layouts
Journal
IF:
2.7
Papers:
1.1K
Citations:
1.0W