.png){kind=icon}
.png){kind=icon}
Return
A Scalable Query-Aware Enormous Database Generator for Database Evaluation
.png){kind=icon}
OA
AI
DOI：10.1109/TKDE.2022.3153651.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Query-aware synthetic data generation is an essential and highly challenging task, important for database management system (DBMS) testing, database application testing and application-driven benchmarking. Prior studies on query-aware data generation suffer common problems of limited parallelization, poor scalability, and excessive memory consumption, making these systems unsatisfactory to terabyte scale data generation. In order to fill the gap between the existing data generation techniques and the emerging demands of enormous query-aware test databases, we design and implement a new data generator, called Touchstone. Touchstone adopts the random sampling algorithm instantiating query parameters and the new data generation schema generating the test database, to achieve fully parallel data generation, linear scalability and austere memory consumption. It has full support of outer joins as well as non-equi-joins for application-oriented data generation. Our experimental results show that Touchstone consistently outperforms the state-of-the-art solution on TPC-H workload by a 1000x speedup without sacrificing simulation fidelity.
Keywords:
Databases
Generators
Memory management
Engines
Benchmark testing
Task analysis
Standards
Query-aware data generator
OLAP database testing
query generator
Journal
IF:
10.4
Papers:
6.7K
Citations:
3.2W