.png){kind=icon}
返回
Efficient Distributed All-Pairs Algorithms: Management Using Optimal Cyclic Quorums
.png){kind=icon}
OA
AI
DOI：10.1109/TPDS.2017.2707417.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
All Pairs problems occur in many research fields. The all-pairs problem requires all data elements to be paired with all other data elements. With the advent of new data intensive big data applications and increase in data size, methods to reduce memory foot print and distribute to work equally across compute nodes are needed. In this paper, we propose cyclic quorum sets for all-pairs algorithm computations to reduce memory foot print. We show that the cyclic quorum sets have a unique all-pairs property that allows for minimal data replication. The cyclic quorums set based computing requires only N/root P size memory, up to 50 percent smaller than the dual N/root P N/root P array implementations proposed earlier, and significantly smaller than solutions requiring all data in each node. Computation can be distributed efficiently and more importantly and are communication-less after initial data distribution, which is a huge advantage in minimizing computation time. Scaling from 16 to 512 cores ( 1 to 32 compute nodes), our application experiments on a real dataset demonstrated scalability with greater than 150x (super-linear) speedup with less than 1/4th the memory usage per node in our experiments.
Keyword:
Distributed all-pair computations
cyclic quorum set and all pair property
communication-less computation
memory foot print efficiency
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6
论文数:
5.2K
被引数:
1.1W
机构
引用论文
分享收藏分享收藏Effects of Fiber Type and Fiber Volume Content on Frost Resistance of Fiber-Reinforced Concrete in Airport Pavement.png)
ICTE 2011
ICTE 2011IF0
PREAI
分享收藏
查看更多内容