.png){kind=icon}
返回
Stream Aggregation with Compressed Sliding Windows
.png){kind=icon}
OA
AI
DOI：10.1145/3590774.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
High performance stream aggregation is critical for many emerging applications that analyze massive volumes of data. Incoming data needs to be stored in a sliding window during processing, in case the aggregation functions cannot be computed incrementally. Updating the window with new incoming values and reading it to feed the aggregation functions are the two primary steps in stream aggregation. Although window updates can be supported efficiently using multi-level queues, frequent window aggregations remain a performance bottleneck as they put tremendous pressure on the memory bandwidth and capacity. This article addresses this problem by enhancing StreamZip, a dataflow stream aggregation engine that is able to compress the sliding windows. StreamZip deals with a number of data and control dependency challenges to integrate a compressor in the stream aggregation pipeline and alleviate the memory pressure posed by frequent aggregations. In addition, StreamZip incorporates a caching mechanism for dealing with skewed-key distributions in the incoming data stream. In doing so, StreamZip offers higher throughput as well as larger effective window capacity to support larger problems. StreamZip supports diverse compression algorithms offering both lossless and lossy compression to integers as well as floating-point numbers. Compared to designs without compression, StreamZip lossless and lossy designs achieve up to 7.5x and 22x higher throughput, while improving the effective memory capacity by up to 5x and 23x, respectively.
Keyword:
Compression
dataflow
aggregation
sliding windows
stream processing
期刊
IF:
2.8
论文数:
598
被引数:
810
机构
引用论文
Assessment of SOAP note evaluation tools in colleges and schools of pharmacy药学学院和药学院校中SOAP记录评估工具的评估
PREAI
分享收藏Evaluating the effectiveness of a peer-led education intervention to improve the patient safety attitudes of junior pharmacy students: a cross-sectional study using a latent growth curve modelling approach评估同伴引导式教育干预对改善低年级药学学生患者安全态度有效性的研究：一项采用潜在增长曲线模型方法的横断面研究.png)
BMJ OpenOAAI
BMJ OpenIF0
OAAI分享收藏
查看更多内容