Evaluating Approaches to Efficient Arbitrage in Serverless Contexts

Abstract

Serverless is a cloud computing paradigm that's enjoyed tremendous growth in popularity in the last five years. Serverless is most known for it's unique pricing model, where clients pay for compute time, rather than allocated VM time. Today, all major cloud providers offer their own version of serverless (also known as Function-as-a-Service). The serverless model is built for compute-heavy, event-driven workloads. Examples of this are encoding video or resizing images as they are uploaded to cloud storage. Serverless infrastructure is built using multiple layers of virtualization to enhance the rapid creation of virtual machines necessary to run ephemeral workloads. We show that these containers (called instances) have a wide variance in performance characteristics. We go on to demonstrate various approaches to ensure a client is guaranteed high performing instances within the demonstrated variance. These approaches were met with varying degrees of success. The most successful approach was using K-means clustering to identify low performing instances. By rejecting these low performers, we were able to gain a 1% increase in performance on average. More aggressive rejection policies resulted in 3% increase in performance on average. Both these policies came at the cost of rejecting a sizable portion of function invocations, which meant more functions needed to be invoked overall. This potential increase in performance is applied to a theoretical real-world use case at the streaming company Netflix. We show our method of arbitrage can potentially save tens of thousands of dollars annually for high volume serverless customers