arrow
Return

Benchmarking Serverless Computing Platforms

delete2020-07-07
delete20
PRE
AI
H
Horácio José Morgado Martins
F
Filipe Araújo
P
Paulo Rupino da Cunha *
DOI:10.1007/s10723-020-09523-1delete
deleteOriginal
deleteShare
deleteSave
Abstract

Abstract

En 中文
We propose a benchmarking test suite to evaluate performance of cloud serverless platforms and an open source software tool to automate the test process. Additionally, we used this setup to compare the commercial offers of Amazon, Google, Microsoft, and IBM. The work builds on ideas and experiments reported in the literature that, nevertheless, did not offer a standard set of coherent and comprehensive tests, capable of analyzing diverse serverless platforms, across the same set of features. To that end, we defined seven tests that cover scalability (latency and throughput), the effect of allocated memory, the performance for CPU-bound cases, the impact of payload size, the influence of the programming language, resource management (e.g., reuse of containers), and overall platform overhead. We created a software tool to deploy the test code and collect metrics in a manner agnostic to the serverless platforms under study. At a time when serverless computing popularity is rising, this benchmarking suite and software test tool enable developers to take informed decisions on the suitability and performance of each provider's serverless offer. Additionally, they also help to identify attributes in need for improvement in the existing platforms.
Keywords:
Cloud computing
Serverless computing
Benchmarking

Journal

Journal of Grid Computing cover
Journal of Grid Computing
IF:
2.9
Papers:
770
Citations:
1.5K

Organization

U
universidade de coimbra
Scholars:
302
Papers: 108
Citations: 8.6K
Cited Papers

Cited Papers

Architecture and self-assembly of the SARS-CoV-2 nucleocapsid protein
err
IF0
err2020-05-17
err0
errOAAI
errQiaozhen Ye; Alan M.V. West; Steve Silletti; Kevin D. Corbett
errShare
errSave
Cost-Performance Modeling with Automated Benchmarking on Elastic Computing Clouds
err2017-10-23
err3
PREAI
errMao, Hongyan; Qi, Zhengwei; Duan, Jiangang; Ge, Xinni
errShare
errSave
A mixed-method empirical study of Function-as-a-Service software development in industrial practice
err2019-03-01
err72
PREAI
errLeitner, Philipp; Wittern, Erik; Spillner, Josef; Hummer, Waldemar
errShare
errSave
Electrically-Transduced Chemical Sensors Based on Two-Dimensional Nanomaterials
err2019-01-09
err0
PREAI
errZheng Meng; Robert M. Stolz; Lukasz Mendecki; Katherine A. Mirica
errShare
errSave
no more