.png){kind=icon}
返回
Improving Oversubscribed GPU Memory Performance in the PyTorch Framework
.png){kind=icon}
PRE
AI
DOI：10.1007/s10586-022-03805-x.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Popular deep learning frameworks like PyTorch utilize GPUs heavily for training, and suffer from out-of-memory (OOM) problems if memory is not managed properly. CUDA Unified Memory (UM) allows the oversubscription of tensor objects in the GPU, but suffers from heavy performance penalties. In this paper, we build upon our UM implementation and create and utilize a minimal overhead CUPTI dynamic profiler to trace unified memory page fault and memory transfer statistics in PyTorch applications. We also implement CUDA memory prefetch and advise API which can be called directly from the PyTorch application based on the dynamically profiled statistics to improve oversubscription performance in various PyTorch models including Resnet and BERT.
Keyword:
CUDA
Unified memory
PyTorch
prefetch
Advise
CUPTI
期刊
C
IF:
4.1
论文数:
5.1K
被引数:
7.5K
机构
引用论文
Matrix Effects in the Detection of Pb and Ba in Soils Using Laser-Induced Breakdown Spectroscopy使用激光诱导击穿光谱检测土壤中Pb和Ba的基质效应
PREAI
分享收藏MHACO: a Multi-Objective Hypervolume-Based Ant Colony Optimizer for Space Trajectory Optimization
OAAI
OAAI分享收藏"Sisterhood and Sociability": The Utah Women’s Press Club, 1891 - 1928《姐妹情谊与社会交往》：犹他州女记者俱乐部，1891 - 1928
PREAI
分享收藏Identification of nodal micrometastasis in colorectal cancer using deep learning on annotation-free whole-slide images.png)
MODERN PATHOLOGYOAAI
MODERN PATHOLOGYIF5.5
OAAI分享收藏
查看更多内容