.png){kind=icon}
返回
Safe Data Parallelism for General Streaming
.png){kind=icon}
OA
AI
DOI：10.1109/TC.2013.221.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Streaming applications process possibly infinite streams of data and often have both high throughput and low latency requirements. They are comprised of operator graphs that produce and consume data tuples. General streaming applications use stateful, selective, and user-defined operators. The stream programming model naturally exposes task and pipeline parallelism, enabling it to exploit parallel systems of all kinds, including large clusters. However, data parallelism must either be manually introduced by programmers, or extracted as an optimization by compilers. Previous data parallel optimizations did not apply to selective, stateful and user-defined operators. This article presents a compiler and runtime system that automatically extracts data parallelism for general stream processing. Data-parallelization is safe if the transformed program has the same semantics as the original sequential version. The compiler forms parallel regions while considering operator selectivity, state, partitioning, and graph dependencies. The distributed runtime system ensures that tuples always exit parallel regions in the same order they would without data parallelism, using the most efficient strategy as identified by the compiler. Our experiments using 100 cores across 14 machines show linear scalability for parallel regions that are computation-bound, and near linear scalability when tuples are shuffled across parallel regions.
Keyword:
Data processing
distributed computing
parallel programming
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.8
论文数:
5.3K
被引数:
9.8K
机构
引用论文
Latitude and Weather Influences on Sun Light Quality and the Relationship to Tree Growth.png)
ForestsOAAI
ForestsIF0
OAAI分享收藏Metal carbonyls—II Carbon monoxide and chloride exchange with some cyclopentadienyl metal carbonyls
PREAI
分享收藏Incidence of Dementia in Relation to Genetic Variants at PITX2, ZFHX3, and ApoE ε4 in Atrial Fibrillation Patients房颤患者中与PITX2，ZFHX3和apoeε4遗传变异相关的痴呆发生率OAAI
OAAI分享收藏
查看更多内容