.png){kind=icon}
返回
Multi-core parallel architecture design and experiment for deep learning model training
.png){kind=icon}
PRE
AI
DOI：10.1007/s11042-022-12292-6.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The parallel architecture can improve the training speed of Deep learning models and it is beneficial to model optimization. This research designs Optimal Interleaved Distributed Architecture (OIDA). Its characteristics are (1) the data set that each child node participates in training is fixed and unique; (2) multiple approaches ensure that the model parameters of each child node participating in training each time are always optimal; (3) the computing units of each child node are interleaved for training without stopping. The architecture can be deployed as Single Machine Parallel (SMP) or Multi Machine Distributed (MMD), or a combination of both. The experimental results show that its efficiency is obvious in a multi-machine multi-card environment, and the model speed of training millions of parameters in a 5-machine 10-card lightweight (Graphics Processing Unit (GPU) of 2GB memory) cluster can reach megapixels/s. When training the same amount of training data, GPU can process data faster than Central Processing Unit (CPU) with the same number of clusters. 4-machine 4-core GPU parallel saves 65.61% of time compared with 4-machine 4-core CPU parallel, while 4-machine 8-core GPU saves 83.32% of time compared with 4-machine 4-core CPU parallel. For polarized Synthetic Aperture Radar (SAR) Deep learning data sets, increasing the number of computers in the cluster can effectively save training time. In the experimental environment of this research, there is no situation that when the number of computers in the cluster increases due to network data transmission, the training time does not decrease.
Keyword:
Deep learning
Model training
Multi-core parallel
Architecture design
期刊
IF:
3
论文数:
2.0W
被引数:
3.2W
机构
暂无机构信息
引用论文
分享收藏Geopolymer - Synthetic Based Mud Hybrid Cements for Primary Cementing and Lost Circulation Control
PREAI
分享收藏Traffic Flow Imputation Using Parallel Data and Generative Adversarial Networks基于并行数据和生成对抗网络的交通流插补
PREAI
分享收藏Palliative superselective intra–arterial chemotherapy for advanced nonresectable gastric cancer姑息性超选择性经动脉化疗治疗进展期不可切除胃癌.png)
IF0
PREAI
分享收藏Phase II study of first–line biweekly docetaxel and cisplatin combination chemotherapy in advanced gastric cancer二线研究：晚期胃癌每周二线多西他赛和顺铂联合化疗方案
IF0
PREAI
分享收藏Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏
查看更多内容