Dynamically Scaling Applications in the Cloud

Presented by Paul

Problem

• The cloud offerso Near infinite computing capabilitieso Near infinite storageo On-demand access to new VMso Pay-as-you-go model

• But how do you build an application to automatically take advantage of those resources?

[1]

Overview of scaling options

[1]

Horizontal Scaling

• VM Replicationo Duplicate servers and create load balancers to

distribute incoming requesto Most common approach to scalingo Will look in more detail...

• Network Scalabilityo As an application scales up, the bandwidth

requirements for intercloud communication and app requests increases, [1] assesses the possibility of requesting network resources in conjunction with cloud resources Network as a Server (NaaS)

[1]

Vertical Scaling

• VM resizing (live)o Taking a VM with given processing power, and

upgrading it with more resources while it's still running

• VM replacement o Replacing less powerful VM with more powerful VM

(shutting down original VM)

• Might require reboots, and application tolerance

[1]

"Rule of thumb"

• CPU-Intensiveo Best to load balance and split computation among

many instances

• Network intensiveo Better to use single CPU heavy instance

• More network intensiveo DNS-based load balancing on powerful CPUs

[1]

Decisions, decisions

• 1 Cloud to rule them all?o Do you want to use multiple cloud providers?o Does part of your application require running on

private cloud?

• How much control over automated scaling would you like?o If you don't want any control, maybe PaaS is better

optiono Do you simply want horizontal scaling, and will

handle load balancing on your own?o How much $$$ do you have?

Methods for Scaling

• Automatic VM Scaling

• Dynamic Workload-pattern Matching

• Whole Application Scaling

• Non-scalable Load Balancing (e.g. Elastic Load Balancing)

• DNS-based Load Balancing

[1]

Basic: Automatic VM Scaling

• Services that scale based on predefined VM-related performance metrics

• Offered by Amazon and Rightscaleo Amazon Autoscaling comes with Cloudwatch

Set conditions based on Cloudwatch variables, react to latency, CPU speed, etc.

• What you might imagine for scaling - like a state machineo If conditions a,b,c are met, do x,y,z

[1]

The "Controller"

"Non-Scalable" Load Balancers

• Amazon offers load balancing service (Elastic Load Balancing)o The reason it's "non-scalable" is this is a single load

balancer

• Could be used in conjunction with horizontal scalingo Scale up/down the number of instances, configure

load balancer to balance across running instances

Elastic Beanstalk

• Automatically scale up your application depending on services it's using, and beanstalk controller parameterso Only pay for underlying AWS resources

• Can upload .NET, PHP/Python, and Java apps for integration with Elastic Beanstalk

• Closer in function to a PaaSo Still have access to AWS instances, can interact with

elastic beanstalk controller

Whole Application Scaling

[3]

Whole Application Scaling

[3]

Dynamic Workload-pattern Matching

[2]Scalability Controller

Dynamic Workload-pattern Matching

[2]

Final Observations

• You must first decide how many clouds you want to use

• Scaling is still an arto Systems for automated scaling still in their infancy

• At the core, there is always a scaling controllero Either you configure and operate the controller, or

the IaaS provider does (e.g. Elastic Beanstalk)

• You pay for automationo Either with your time or your moneyo More customized automation == more complexity

Sources

[1] L. M. Vaquero, L. Rodero-Merino, and R. Buyya. "Dynamically scaling applications in the cloud," SIGCOMM Comput. Commun. Rev., 41:45-52.

[2] P. Marshall, K. Keahey, and T. Freeman, "Elastic site:Using clouds to elastically extend site resources," Cluster Computing and the Grid, IEEE International Symposium on, vol. 0, pp. 43-52, 2010.

[3]R. Buyya, R. Ranjan, and R. Calheiros, “Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services.” in ICA3PP 2010: The 10th International Conference on Algorithms and Architectures for Parallel Processing, 2010, pp. 19–24.

Links for papers

[1] http://www.cloudbus.org/papers/ScalabilityInCloud2011.pdf

[2] http://www.nimbusproject.org/files/elasticsite_ccgrid_2010.pdf

[3] http://arxiv.org/pdf/1003.3920.pdf