Elastic Scaling for Data Stream Processing

Buğra Gedik

doi:10.1109/TPDS.2013.295

2014-06-01

168

PRE

DOI：10.1109/TPDS.2013.295

原文链接

原文求助

摘要

En 中文

This article addresses the profitability problem associated with auto-parallelization of general-purpose distributed data stream processing applications. Auto-parallelization involves locating regions in the application's data flow graph that can be replicated at run-time to apply data partitioning, in order to achieve scale. In order to make auto-parallelization effective in practice, the profitability question needs to be answered: How many parallel channels provide the best throughput? The answer to this question changes depending on the workload dynamics and resource availability at run-time. In this article, we propose an elastic auto-parallelization solution that can dynamically adjust the number of channels used to achieve high throughput without unnecessarily wasting resources. Most importantly, our solution can handle partitioned stateful operators via run-time state migration, which is fully transparent to the application developers. We provide an implementation and evaluation of the system on an industrial-strength data stream processing platform to validate our solution.

Keyword:

Data stream processing

parallelization

elasticity

AI总结

对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。

期刊

IEEE Transactions on Parallel and Distributed Systems

IF:

论文数:

5.2K

被引数:

1.1W

机构

ihsan dogramaci bilkent university

学者数:

3.6K

论文数: 3.5K

被引数: 8

international business machines (ibm)

学者数:

5.7K

论文数: 4.5K

被引数: 4

引用论文

IF0

PREAI

Metal carbonyls—II Carbon monoxide and chloride exchange with some cyclopentadienyl metal carbonyls

Journal of Inorganic and Nuclear Chemistry

IF0

1961-04-01

PREAI

A. Wojcicki; F. Basolo

Testing chemotherapeutic combinations in the human tumor colony—forming assay

Journal of Surgical Oncology

IF0

2006-07-19

PREAI

V. K. Sondak; E. L. Korn; D. L. Morton; D. H. Kern

Rowing Injuries

Sports Health: A Multidisciplinary Approach

IF0

2012-04-26

OAAI

Timothy M. Hosea; Jo A. Hannafin

Multiple-mouse magnetic resonance imaging with cryogenic radiofrequency probes for evaluation of brain development

NeuroImage

IF0

2022-05-01

OAAI

A. Arbabi; L. Spencer Noakes; D. Vousden; J. Dazai; S. Spring; O. Botelho; T. Keshavarzian; M. Mattingly; J.E. Ellegood; L.M.J. Nutter; R. Wissmann; J.G. Sled; J.P. Lerch; R.M. Henkelman; B.J. Nieman

A case of QARS1 associated epileptic encephalopathy and review of epilepsy in aminoacyl-tRNA synthetase disorders

Brain and Development

IF0

2022-02-01

OAAI

Denise L. Chan; Joëlle Rudinger-Thirion; Magali Frugier; Lisa G. Riley; Gladys Ho; Kavitha Kothur; Shekeeb S. Mohammad

Magnetization of Compounds of Rare Earths with Platinum Metals

Physical Review

IF0

1959-09-15

PREAI

R. M. Bozorth; B. T. Matthias; H. Suhl; E. Corenzwit; D. D. Davis

Biologically Inspired Soft Robot for Thumb Rehabilitation1

Journal of Medical Devices

IF0

2014-04-28

PREAI

Paxton Maeder-York; Tyler Clites; Emily Boggs; Ryan Neff; Panagiotis Polygerinos; Dónal Holland; Leia Stirling; Kevin Galloway; Catherine Wee; Conor Walsh

Synthesis and Crystal Structure of New Sulfate-Bridged Coordination Polymer, {(4,4′-bpyH2)[Fe3(4,4′-bpy)3(SO4)4(H2O)6]·10H2O}n (4,4′-bpy = 4,4′-Bipyridine). Three-Dimensional Network with Microporous Channels

Chemistry Letters

IF0

2003-09-01

PREAI

Shin-ichiro Noro; Mitsuru Kondo; Susumu Kitagawa; Tomohiko Ishii; Hiroyuki Matsuzaka

Phase 2 study of anastrozole in recurrent estrogen (ER)/progesterone (PR) positive endometrial cancer: The PARAGON trial – ANZGOG 0903

Gynecologic Oncology

IF0

2019-07-01

OAAI

Linda Mileshkin; Richard Edmondson; Rachel L. O'Connell; Katrin M. Sjoquist; John Andrews; Rema Jyothirmayi; Philip Beale; Tony Bonaventura; Jeffrey Goh; Marcia Hall; Andrew Clamp; John Green; Rosemary Lord; Frédéric Amant; Laura Alexander; Karen Carty; James Paul; James Scurry; David Millan; Steven Nottley; Michael Friedlander

查看更多内容