.png){kind=icon}
返回
Enhancing iteration performance on distributed task-based workflows
.png){kind=icon}
OA
AI
DOI：10.1016/j.future.2023.07.032.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Task-based programming models have proven to be a robust and versatile way to approach development of applications for distributed environments. They provide natural programming patterns with high performance. However, execution on this paradigm can be very sensitive to granularity -i.e., the quantity and execution length of tasks. Granularity is often linked with the block size of the data, and finding the optimal block size has several challenges, as it requires inner knowledge of the computing environment.Our proposal is to supplement the task-based programming model with a new mechanism -our SplIter proposal. At its core, the SplIter provides a transparent way to split a collection into partitions (logical groups of blocks, obtained without any transfers nor data rearrangement), which can then be iterated. Tasks are linked to those partitions, which means that SplIter breaks the dependency between block size and task granularity.The evaluation shows that the SplIter is able to achieve performance improvements of over one order of magnitude when compared to the baseline, and it is either competitive or strictly better (depending on application characteristics) to the competitor alternative. We have chosen different applications covering a wide variety of scenarios; those applications are representatives of a broader set of applications and domains. The changes required in the source code of a task-based application are minimal, preserving the high programmability of the programming model. Two different state-of-the-art task-based frameworks have been evaluated for all the applications: COMPSs and Dask, showing that the SplIter can be effectively used within different frameworks.& COPY; 2023 Elsevier B.V. All rights reserved.
Keyword:
Task-based workflows
Distributed computing
Object store
Active storage
Dataset iteration
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
F
IF:
6.1
论文数:
6.9K
被引数:
2.3W
机构
引用论文
.png){kind=icon}
查看更多内容