.png){kind=icon}
.png){kind=icon}
Return
Exploiting Stragglers in Distributed Computing Systems With Task Grouping
.png){kind=icon}
PRE
AI
DOI：10.1109/TSC.2024.3495513.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We consider the problem of stragglers in distributed computing systems. Stragglers, which are compute nodes that unpredictably slow down, often increase the completion times of tasks. One common approach to mitigating stragglers is work replication, where only the first completion among replicated tasks is accepted, discarding the others. However, discarding work leads to resource wastage. In this article, we propose a method for exploiting the work completed by stragglers rather than discarding it. The idea is to increase the granularity of the assigned work, and to increase the frequency of worker updates. We show that the proposed method reduces the completion time of tasks via experiments performed on a simulated cluster as well as on Amazon EC2 with Apache Hadoop.
Keywords:
Clustering algorithms
Standards
Prevention and mitigation
Optimization
Hardware
Stochastic processes
Online services
Encyclopedias
Approximation algorithms
Proposals
Distributed systems
stragglers
task scheduling
Journal
IF:
5.8
Papers:
2.1K
Citations:
6.5K