Efficient erasure-coded data updates based on file class predictions and hybrid writes

doi:10.1016/j.compeleceng.2022.108441

Efficient erasure-coded data updates based on file class predictions and hybrid writes

2022-12-01

1

PRE

AI

DOI：10.1016/j.compeleceng.2022.108441

Original

Original request for help

Save

Abstract

En 中文

A small update write can lead to a partial write to an erasure coding group in erasure-coded storage systems, resulting in a time-consuming write-after-read. This paper presents a data delta and logging based writing approach, named DETOG, to achieve fast partial writes. DETOG classifies files into non-write-only and write-only using machine learning, then adopts different write schemes to perform partial writes according to file classes. For non-write-only files, DETOG uses data deltas that are the differences between latest data values and original data values, instead of parity deltas to reconstruct the failed data. For non-write-only files, DETOG performs in-place data updates and log-based parity updates, so as to balance the overhead of data updates and reads. We implement DETOG on the newly designed prototype storage system to perform performance evaluation. The results of extensive experiments on real-world traces showed that DETOG significantly improved system throughput compared with state-of-the-art methods.

Keywords:

Erasure coding

Efficient partial writes

Machine learning

File classifications

Data delta-based hybrid writes

Efficient erasure-coded data updates based on file class predictions and hybrid writes

Abstract

Journal

Organization