.png){kind=icon}
.png){kind=icon}
Return
Deterministic Data Distribution for Efficient Recovery in Erasure-Coded Storage Systems
.png){kind=icon}
OA
AI
DOI：10.1109/TPDS.2020.2987837.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Due to individual unreliable commodity components, failures are common in large-scale distributed storage systems. Erasure codes are widely deployed in practical storage systems to provide fault tolerance with low storage overhead. However, random data distribution (RDD), commonly used in erasure-coded storage systems, induces heavy cross-rack traffic, load imbalance, and random access, which adversely affects failure recovery. In this article, with orthogonal arrays, we define a Deterministic Data Distribution ($D<^>3$D3) to uniformly distribute data/parity blocks among nodes, and propose an efficient failure recovery approach based on $D<^>3$D3, which minimizes the cross-rack repair traffic against a single node failure. Thanks to the uniformity of $D<^>3$D3, the proposed recovery approach balances the repair traffic not only among nodes within a rack but also among racks. We implement $D<^>3$D3 over Reed-Solomon codes and Locally Repairable Codes in Hadoop Distributed File System (HDFS) with a cluster of 28 machines. Compared with RDD, our experiments show that $D<^>3$D3 significantly speeds up the failure recovery up to 2.49 times for RS codes and 1.38 times for LRCs. Moreover, $D<^>3$D3 supports front-end applications better than RDD in both of normal and recovery states.
Keywords:
Distributed storage system
erasure coding
traffic
orthogonal array
load balance
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6
Papers:
5.2K
Citations:
1.1W