FaaSCtrl: A Comprehensive-Latency Controller for Serverless Platforms

Abhisek Panda

doi:10.1109/TCC.2024.3473015

FaaSCtrl: A Comprehensive-Latency Controller for Serverless Platforms

2024-10-01

0

PRE

AI

DOI：10.1109/TCC.2024.3473015

Original

Original request for help

Save

Abstract

En 中文

Serverless computing systems have become very popular because of their natural advantages with respect to auto-scaling, load balancing and fast distributed processing. As of today, almost all serverless systems define two QoS classes: best-effort (BE) and latency-sensitive (LS). Systems typically do not offer any latency or QoS guarantees for BE jobs and run them on a best-effort basis. In contrast, systems strive to minimize the processing time for LS jobs. This work proposes a precise definition for these job classes and argues that we need to consider a bouquet of performance metrics for serverless applications, not just a single one. We thus propose the comprehensive latency (CL) that comprises the mean, tail latency, median and standard deviation of a series of invocations for a given serverless function. Next, we design a system FaaSCtrl, whose main objective is to ensure that every component of the CL is within a prespecified limit for an LS application, and for BE applications, these components are minimized on a best-effort basis. Given the sheer complexity of the scheduling problem in a large multi-application setup, we use the method of surrogate functions in optimization theory to design a simpler optimization problem that relies on performance and fairness. We rigorously establish the relevance of these metrics through characterization studies. Instead of using standard approaches based on optimization theory, we use a much faster reinforcement learning (RL) based approach to tune the knobs that govern process scheduling in Linux, namely the real-time priority and the assigned number of cores. RL works well in this scenario because the benefit of a given optimization is probabilistic in nature, owing to the inherent complexity of the system. We show using rigorous experiments on a set of real-world workloads that FaaSCtrl achieves its objectives for both LS and BE applications and outperforms the state-of-the-art by 36.9% (for tail response latency) and 44.6% (for response latency's std. dev.) for LS applications.

Keywords:

Measurement

Tail

Quality of service

Standards

Optimization

Reinforcement learning

Real-time systems

Degradation

Schedules

Job colocation

serverless computing

performance interference

reinforcement learning

resource scheduling

FaaSCtrl: A Comprehensive-Latency Controller for Serverless Platforms

Abstract

Journal

Organization

Cited Papers