.png){kind=icon}
返回
Collaborative deep learning across multiple data centers
.png){kind=icon}
PRE
AI
DOI：10.1007/s11432-019-2705-2.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data of different organizations to a centralized data center owing to the constraints of privacy regulations. It is very challenging to conduct the geo-distributed deep learning among data centers without the privacy leaks. Model averaging is a conventional choice for data parallelized training and can reduce the risk of privacy leaks, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model averaging can be effective in the decentralized environment by using two strategies, namely, the cyclical learning rate (CLR) and the increased number of epochs for local model training. With the two strategies, we show that model averaging can provide competitive performance in the decentralized mode compared to the data-centralized one. In a practical environment with multiple data centers, we conduct extensive experiments using state-of-the-art deep network architectures on different types of data. Results demonstrate the effectiveness and robustness of the proposed method.
Keyword:
collaborative learning
multiple datacenters
distributed machine learning
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.6
论文数:
4.9K
被引数:
8.9K
机构
引用论文
Use of SIMD Vector Operations to Accelerate Application Code Performance on Low-Powered ARM and Intel Platforms.png)
2013 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum
OAAI
2013 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd ForumIF0
OAAI分享收藏分享收藏Catalytic activity of dihydride ruthenium complexes in the hydrogenation of nitrogen containing heterocycles二氢化物钌配合物在含氮杂环氢化中的催化活性
PREAI
分享收藏
Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏
查看更多内容