PERFORMANCE MODEL DRIVEN QOS GUARANTEES AND

OPTIMIZATION IN CLOUDSJohn Chinneck

Carleton University

Marin Litoiu

York University

Gabriel Iszlai

IBM

ICSE Workshop on Software Engineering Challenges of Cloud Computing

Jim (Zhanwen) Li

Carleton University

Presented by: Yun Liaw

Outline

Introduction Cloud Architecture Engineering for QoS Optimization Case Study Conclusions and Comments

112/04/19

2

Introduction

Cloud management is responsible for all resources used by all applications deployed in the cloud

The opportunity for global resource optimization is a major driver for Cloud implementation

112/04/19

3

Introduction

Each application and resource have their own price, thus the profit of the application provider (AP) and cloud provider (AP) can thus be calculated

But QoS is another goal that needs to be achieved of cloud management – constraint of optimization

For simplicity, this paper only consider the response time as the QoS parameter

This paper assumes that:“When the total AP profits are maximized, the CP can arrange that its profit is also maximized”

112/04/19

4

Cloud Architecture

The three-level hierarchy Cloud Architecture Application developer: tunes the

code over time, discovering and improving inefficient operations

AP admin: tunes the runtime configuration to make the best use of the existing resources

CP admin: physical resource allocation

112/04/19

5

Service Architecture

Assume that deployable unit is the concurrent process, termed a task

A service model comprises UserClass, Service, ServerTask, Resource and Host UserClass raises request from outside Services request each other inside and outside of the

service, forming a web of inter-service traffic Service are implemented by Applications which run as

system tasks or thread pools (ServerTasks)

112/04/19

6

Service Architecture

112/04/19

7

A simplified metamodel for a service architecture

Service Architecture Represented by LQM

112/04/19

8

LQM: Layered Queuing Model

Host processor

The Performance Model

The role of performance model To predict the effect of simultaneous changes to many decision

variables To assist in making optimal decisions over many variables

Layered Queuing Model [*] A kind of extended queuing network Can be solved to predict:

throughputs, mean queuing delays, mean service delays at entries, and resource utilization

Benefit of using LQM here: corresponds to architecture, represents the layered resource behavior

112/04/19

9

[*] G. Franks, et al., “Layered Bottlenecks and their mitigation,” Proc. Int. Conf. on Quantitative Evaluation of Systems, 2007[*] G. Franks, et al., “Enhanced Modeling and Solution of Layered Queuing Networks,” IEEE Trans. On Softw. Eng., 2008[*] J. Rolia, et al., “The Method of Layer,” IEEE Trans. Softw. Eng., 1995

Workload and QoS Requirement

Workload describes the intensity of the streams of user requests for service, in terms of throughput fc for user class c, or the number Nc of users that are interacting and their think time

Zc Zc represents the user’s mean delay between receiving a response,

and issuing its next request Assume that each service has its own service class c, with

Nc users and think time Zc sec The QoS requirement of each class can be defined by:

Throughput fc, min

Response time: Rc, max

By Little’s Law, fc f≧ c,min = Nc / (Rc, max + Zc)

112/04/19

10

Little’s Law: L = λ×W

Queue length Arrival rate (throughput)

Waiting time

The Optimization Loop

112/04/19

11

Engineering for QoS and Optimization The design requirements of Cloud:

The application software should be efficient and adaptable to different run-time situation

The cloud must provide infrastructure for deploying and monitoring application elements and user QoS

The cloud feedback loop must be able to track the performance model and to optimize the management decisions

112/04/19

12

Developer Responsibility

Mostly the cloud hides resource management from the developer, but for QoS, efficient execution is the developer’s responsibility

Another subtle goal for the developer is to provide flexibility in the concurrency architecture (the allocation of functions to tasks) Concurrency creates flexibility, but introduces overheads

The developer also needs to provide the structure of the performance model E.g., the service’s component interact diagrams

112/04/19

13

112/04/1914The model-based optimization architecture

The estimation tool for updating the model param. periodically

Optimization technique

Network Flow Model (NFM) Shows the flows of execution demands (in CPU-sec per

sec) at the processors And how they combine to produce flows at the tasks,

services and user-responses Nodes in NFM:

Nodes for processors have a flow equal to the processor utilization

Nodes for tasks have flows from all processors on which the task is deployed

Nodes for user classes have flows from the tasks which implements the service

112/04/19

15

NFM Example

112/04/19

16

[X,Y,Z]X: the lower flow boundY: the upper flow boundZ: the cost per flow unit

The demand rates

The maximum demand rate available at host h – host h’s capacity

[*] Z. Li, et al., “Fast Scalable Optimization to Configure Service Systems having Cost and Quality of Service Constraints,” in Proc. International Conference on Autonomic Computing, 2009

Demand rate of a request issued by user

Fc = γsc/dsc

Optimization technique

112/04/19

17

Objective Function

Each Service class c offered to users has a price per response of Pc

Each host h has a price of CPU execution of Ch per CPU-sec, including unused CPU-sec allocated in order to reduce contention delays

Each task t has a reservation αht, in CPU-sec per sec, on some host h

The Application App has a profit function:

112/04/19

18

AppApp inth hthinc ccApp CfPPROFIT

, ψApp: sets of user classesτApp: sets of tasks

Objective Function

Remark that this paper assumed that if the AP’s profit is maximized, the CP’s profit is also maximized

Constraints of the objective function: A maximum user response time Rc, max for each class c, or a

minimum class throughput fc, min

A minimum profit PROFITapp, min for each application

112/04/19

19

App AppPROFITTOTAL

AppApp inth hthinc ccApp CfPPROFIT

,

Case Study

A Cloud with VMs and from 1 to 50 application instances Applications are the same but with different parameters

Two deployment scenarios: “Incremental” deployment scenario: Place each application

on a sufficient number of processors to meet its profit and QoS constraints, disjoint from already allocated processors

“full” deployment scenario: The deployment is optimized the overall profit based on the previously mentioned objective function Pros: The full deployment can maximize the profit Cons: When the new instance is created, the existing instance

also needs to be re-deployed

112/04/19

20

Case Study - result

112/04/19

21

Conclusion & Comments

Conclusion An approach of resource allocation optimization is summarized This approach is effective and scalable, and the prototype has

been implemented Further development: take into account for memory allocation,

communication delays, VM overhead costs, and licensing costs of software replicas

Comments: The assumption related to the CP profit does not verified No much details described References of this paper (performance models) might be valuable

for my ongoing work

112/04/19

22