.png){kind=icon}
返回
Data-Aware Adaptive Compression for Stream Processing
.png){kind=icon}
PRE
AI
DOI：10.1109/TKDE.2024.3377710.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Stream processing has been in widespread use, and one of the most common application scenarios is SQL query on streams. By 2021, the global deployment of IoT endpoints reached 12.3 billion, indicating a surge in data generation. However, the escalating demands for high throughput and low latency in stream processing systems have posed significant challenges due to the increasing data volume and evolving user requirements. We present a compression-based stream processing engine, called CompressStreamDB, which enables adaptive fine-grained stream processing directly on compressed streams, to significantly enhance the performance of existing stream processing solutions. CompressStreamDB utilizes nine diverse compression methods tailored for different stream data types and integrates a cost model to automatically select the most efficient compression schemes. CompressStreamDB provides high throughput with low latency in stream SQL processing by identifying and eliminating redundant data among streams. Our evaluation demonstrates that CompressStreamDB improves average performance by 3.84x and reduces average delay by 68.0% compared to the state-of-the-art stream processing solution for uncompressed streams, along with 68.7% space savings. Besides, our edge trials show an average throughput/price ratio of 9.95x and a throughput/power ratio of 7.32x compared to the cloud design.
Keyword:
Encoding
Compression algorithms
Real-time systems
Low latency communication
Data processing
Costs
Delays
Data compaction and compression
stream processing
edge computing
期刊
IF:
10.4
论文数:
6.9K
被引数:
3.2W
机构
引用论文
Strategy of Digital Education Resources Construction in Cloud-Computing Environment.png)
2016 Eighth International Conference on Measuring Technology and Mechatronics Automation (ICMTMA)
2016 Eighth International Conference on Measuring Technology and Mechatronics Automation (ICMTMA)IF0
PREAI
分享收藏The presence of mast cells in the human endometrium pre- and post-insertion of intrauterine devices
PREAI
分享收藏Reliability and construct validity of the Automated Neuropsychological Assessment Metrics (ANAM) mood scale自动神经心理学评估指标 (ANAM) 情绪量表的信度和结构效度
OAAI
OAAI分享收藏分享收藏分享收藏
查看更多内容