.png){kind=icon}
Return
Using Microbenchmark Suites to Detect Application Performance Changes
.png){kind=icon}
OA
AI
DOI：10.1109/TCC.2022.3217947.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Software performance changes are costly and often hard to detect pre-release. Similar to software testing frameworks, either application benchmarks or microbenchmarks can be integrated into quality assurance pipelines to detect performance changes before releasing a new application version. Unfortunately, extensive benchmarking studies usually take several hours which is problematic when examining dozens of daily code changes in detail; hence, trade-offs have to be made. Optimized microbenchmark suites, which only include a small subset of the full suite, are a potential solution for this problem, given that they still reliably detect the majority of the application performance changes such as an increased request latency. It is, however, unclear whether microbenchmarks and application benchmarks detect the same performance problems and one can be a proxy for the other. In this paper, we explore whether microbenchmark suites can detect the same application performance changes as an application benchmark. For this, we run extensive benchmark experiments with both the complete and the optimized microbenchmark suites of the two time-series database systems InfluxDB and VictoriaMetrics and compare their results to the results of corresponding application benchmarks. We do this for 70 and 110 commits, respectively. Our results show that it is possible to detect application performance changes using an optimized microbenchmark suite if frequent false-positive alarms can be tolerated.
Keywords:
Benchmark testing
Codes
Cloud computing
Reliability
Software
Pipelines
MIMICs
Benchmarking
microbenchmarks
performance testing
performance change detection
regression detection
Journal
I
IF:
5
Papers:
1.9K
Citations:
4.3K
Organization
Cited Papers
ShareSaveShareSaveShareSaveFacile preparation of in situ coated Ti3C2Tx/Ni0.5Zn0.5Fe2O4composites and their electromagnetic performance.png)
RSC Advances
OAAI
RSC AdvancesIF0
OAAIShareSaveShareSave
View more