Multi-tenant Resource Allocation For Shared Cloud Storage

Abstract

Shared storage services enjoy wide adoption in commercial clouds. But few systems today provide performance isolation or resource allocation. Misbehaving or high-demand tenants can overload the shared service and disrupt other well-behaved tenants, leading to unpredictable performance and violating tenant service level agreements. Today's approaches for multi-tenant resource allocation are based either on per-virtual machine allocations or hard rate limits that assume uniform workloads to achieve high utilization.

In this thesis, we present a framework for achieving datacenter-wide per-tenant performance isolation and fairness in a shared key-value storage system we call Pisces. Pisces achieves per-tenant weighted fair shares of aggregate system resources, even when different tenants' partitions are co-located and when partition demand is skewed, time-varying, or bottlenecked by different server resources. Pisces does so by decomposing the fair sharing problem into four complementary mechanisms --partition placement, weight allocation, replica selection, and weighted fair queuing--that operate on different time-scales and combine to provide system-wide max-min fairness. Our Pisces storage prototype achieves nearly ideal (0.99 Min-Max Ratio) fair shares, strong performance isolation, and robustness to skew and shifts in tenant demand at high utilization.

Although Pisces achieves resource sharing for network-bound workloads, such guarantees have proven elusive for disk-bound workloads. Modern storage stacks often amplify the IO cost of application requests. Interference between tenant IO workloads suppresses throughput and induces variability. Even without interference, IO performance varies non-linearly with operation size. To address these challenges, we present Libra, an IO scheduling framework that provides per-tenant application-request throughput reservations while achieving high utilization for SSD-based storage. Libra tracks per-request IO consumption to provision IO resources based on tenants' dynamic usage profiles. Using an IO cost model based on virtual IO operations (VOP), Libra can allocate and schedule tenant IO within the limits of provisionable IO throughput. Our LevelDB-based prototype achieves highly accurate tenant app-request reservations and low-level VOP allocations (> 0.95 Min-Max Ratio) over a range of workloads with high utilization.