.png){kind=icon}
返回
Off-Chip Memory Allocation for Neural Processing Units
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3352900.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Many modern Systems-on-Chip (SoCs) are equipped with specialized Machine Learning (ML) accelerators that use both on-chip and off-chip memory to execute neural networks. While on-chip memory usually has a hard limit, off-chip memory is often considered large enough to hold the network's inputs, outputs, weights, and any intermediate results that may occur during model execution. This assumption may not hold for edge devices, such as smartphones, which usually have a limit on the amount of memory a process can use. In this study, we propose a novel approach for minimizing a neural network's off-chip memory usage by introducing a tile-aware allocator capable of reusing memory occupied by parts of a tensor before the entire tensor expires. We describe the necessary conditions for such an off-chip memory allocation approach and provide the results, showing that it can save up to 33% of the peak off-chip memory usage in some common network architectures.
Keyword:
Tensors
Memory management
Resource management
Neural networks
Mathematical models
Deep learning
System-on-chip
NPU
memory allocation
neural network runtime
tiling
strip-packing problem
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
暂无机构信息
引用论文
Types of Parent Verbal Responsiveness That Predict Language in Young Children With Autism Spectrum Disorder预测自闭症谱系障碍幼儿语言的父母言语反应类型
OAAI
OAAI分享收藏Machine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods机器生成的文本: 威胁模型和检测方法的综合综述.png)
IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏Damage Detection and Level Classification of Roof Damage After Typhoon Faxai Based on Aerial Photo and Deep Learning基于航拍照片和深度学习的Faxai台风后顶板损伤检测与等级分类OAAI
IF0
OAAI分享收藏A novel and efficient xanthenic dye–organometallic ion‐pair complex for photoinitiating polymerization一种用于光引发聚合的新型高效的黄原胶染料-有机金属离子对配合物
PREAI
分享收藏
查看更多内容