.png){kind=icon}
返回
NetEC: Accelerating Erasure Coding Reconstruction With In-Network Aggregation
.png){kind=icon}
OA
AI
DOI：10.1109/TPDS.2022.3145836.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In distributed storage systems, Erasure Coding (EC) is a crucial technology to enable high data availability. By downloading parity data from survived machines, EC can reconstruct lost data with much lower storage overheads than data replication. However, this reduction in storage cost comes at the expense of extra performance problems: low reconstruction rate, high degraded read latency, and high host CPU utilization. Our analysis shows that these performance problems are deeply rooted in the host-based EC processing. To resolve these problems, we present NetEC, an in-network accelerating framework that fully offloads EC to the new generation programmable switching ASICs. We propose Explicit Buffer Size Notification (EBSN) to constrain decoding buffer usage, and design an on-switch one-to-many TCP proxy to integrate EBSN with TCP. We also design two parallel Galois Field (GF) offloading methods-table lookup and bitmatrix methods-to maximize parsable bytes. We implement NetEC on programmable switches and integrate it with HDFS. Extensive evaluations show that NetEC improves the reconstruction rate by 2.7x-6.8x, reduces the degraded read latency significantly, and removes the host CPU overhead completely. We also emulate multi-rack scenarios and show that NetEC is able to support similar to GB/s reconstruction rate and tens of concurrent tasks.
Keyword:
Erasure coding
distributed storage sytems
programmable switch
software-defined networks
期刊
IF:
6
论文数:
5.2K
被引数:
1.1W
机构
引用论文
Hydrogenation of cyclohexene catalyzed by ruthenium nitrosyl complexes: Crystal structures of catalyst precursors [Cp∗Ru(μ2-NO)2RuCp∗] and [Cp∗Ru(NO)(η2-C6H10)] (Cp∗= η5-C5(CH3)5).png)
Polyhedron
PolyhedronIF0
PREAI
分享收藏分享收藏
查看更多内容