Customizable Scale-Out Key-Value Stores

doi:10.1109/TPDS.2020.2982640

Customizable Scale-Out Key-Value Stores

2020-09-01

6

OA

AI

DOI：10.1109/TPDS.2020.2982640

Original

Original request for help

Save

Abstract

En 中文

Enterprise KV stores are often not well suited for HPC applications, and thus cumbersome end-to-end KV design customization is required to meet the needs of modern HPC applications. To this end, in this article we present bespoKV, an adaptive, extensible, and scale-out KV store framework. bespoKV decouples the KV store design into the control plane for distributed management and the data plane for local data store. For the control plane, bespoKVprovides pre-built modules, called controlets, supporting common distributed functionalities (e.g., replication, consistency, and topology) and their various combinations. This decoupling allows bespoKV to take a user-provided single-server KV store, called a datalet, and transparently enables a scalable and fault-tolerant distributed KV store service. The resulting distributed stores are also adaptive to consistency or topology requirement changes and can be easily extended for new types of services. Such specializations enable innovative uses of KV stores in HPC applications, especially for emerging applications that utilize KV-friendly workloads. We evaluate bespoKV in a local testbed as well as in a public cloud settings. Experiments show that bespoKV-enabled distributed KV stores scale horizontally to a large number of nodes, and performs comparably and sometimes 1.2x to 2.6x better than the state-of-the-art systems.

Keywords:

Topology

Distributed databases

Peer-to-peer computing

Fault tolerance

Fault tolerant systems

Nonvolatile memory

Cloud computing

Key-value stores

HPC KV stores

scale-out KV stores

application tailored storage

Customizable Scale-Out Key-Value Stores

Abstract

Journal

Organization