.png){kind=icon}
.png){kind=icon}
Return
MEIS: Optimizing deduplication system with efficient index structure
.png){kind=icon}
PRE
AI
DOI：10.1016/j.sysarc.2025.103620.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The exponential growth of digital data is a critical issue in storage systems as a large amount of duplicate data in the storage systems exerts an extra load on it. Therefore, the deduplication for storage systems has received significant interest in the research community and is increasingly accepted as an effective technique to reduce storage costs. In the deduplication system, the index structure is an efficient solution to improve the performance, although many researchers have proposed to optimize the index structure to cut down its memory overhead, they still have higher memory overheads with the further growth of data scale in the era of big data. Moreover, the optimized index structure can decrease index entry’s insertion and query performance due to the extra computation. To further reduce the memory overhead of the index structure and improve its insertion and query performance, we propose a memory-efficient index structure named MEIS for the deduplication system in this paper, which occupies lightweight memory space with higher performance of insertion and query. The basic idea of MEIS is that both the pointer and the upper layer are eliminated in the binary index tree, leaving only a few leaf nodes, using the bit string parsing mechanism proposed in this paper. To reduce the memory overhead of MEIS while achieving higher insertion and query performance further, a partitioned Huffman coding strategy and a partition bitmap index method are proposed to reduce the index memory overhead of each index entry to 1 bit in the worst case in this paper. Extensive experiments are conducted to prove the effectiveness of MEIS. The experimental results show that the existing methods can bring higher memory overhead than MEIS by 4–5.8 times, and the insertion and query performance of MEIS is 3.5-2251 times higher than that of existing methods.
Journal
IF:
4.1
Papers:
2.9K
Citations:
4.2K
Organization
View more organizations