.png){kind=icon}
.png){kind=icon}
Return
Enabling Fast Exploratory Analyses Over Voluminous Spatiotemporal Data Using Analytical Engines
.png){kind=icon}
OA
AI
DOI：10.1109/TBDATA.2019.2939834.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Fueled by the proliferation of IoT devices and increased adoption of sensing environments the collection of spatiotemporal data has exploded in recent years. Disk based storage systems provide reliable archives but are far too slow for efficient analytics. Furthermore, spatiotemporal datasets quickly exceed the memory capacity of cluster environments. Current solutions focused on inmemory analytics suffer from memory contention and unnecessary network I/O, failing to provide a suitable platform for iterative, exploratory analytics in shared environments. In this work we propose Anamnesis, the first in-memory, sketch aligned, HDFS compliant storage system. Data sketching algorithms reduce dataset sizes by summarizing feature values and inter-feature relationships. Anamnesis leverages data sketches to alleviate memory contention and vastly reduce network I/O during analytics. Upon request, we generate accurate full-resolution datasets with negligible resource and time costs. Datasets are available using a fully HDFS compliant interface allowing Anamnesis to achieve unprecedented compatibility with popular analytics engines. This facilitates adoption into existing workflows by serving as a drop-in replacement for canonical HDFS. We evaluate the system using 2 spatiotemporal datasets, a variety of popular analytics engines, and real-world analytical operations.
Keywords:
Data sketches
spatiotemporal data
big data
distributed analytics
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
I
IF:
5.7
Papers:
860
Citations:
3.0K