.png){kind=icon}
返回
Soft Memory Box: A Virtual Shared Memory Framework for Fast Deep Neural Network Training in Distributed High Performance Computing
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2018.2834146.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep learning is one of the major promising machine learning methodologies. Deep learning is widely used in various application domains, e.g., image recognition, voice recognition, and natural language processing. In order to improve learning accuracy, deep neural networks have evolved by: 1) increasing the number of layers and 2) increasing the number of parameters in massive models. This implies that distributed deep learning platforms need to evolve to: 1) deal with huge/complex deep neural networks and 2) process with high-performance computing resources for massive training data. This paper proposes a new virtual shared memory framework, called Soft Memory Box (SMB), which enables sharing the memory of remote node among distributed processes in the nodes so as to improve communication performance via parameter sharing. According to data-intensive performance evaluation results, the communication time of deep learning using the proposed SMB is 2.1 times faster than that using the massage passing interface (MPI). In addition, the communication time of the SMB-based asynchronous parameter update becomes 2-7 times faster than that using the MPI depending on deep learning models and the number of deep learning workers.
Keyword:
High performance computing
distributed computing
soft memory box
shared memory
deep neural network
distributed deep learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
Structural study of lanthanides(III) in aqueous nitrate and chloride solutions by EXAFS通过EXAFS对硝酸盐和氯化物水溶液中镧系元素 (III) 的结构研究
PREAI
分享收藏3D Printed Electronics With High Performance, Multi-Layered Electrical Interconnect.png)
IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks一种基于递归神经网络的深度学习入侵检测方法IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏
查看更多内容