Cloud Computing Lec7

Basics of Cloud Computing – Lecture 7

Cloud Computing – Summary,

Aneka, some more PaaS and

leftoversleftovers

Satish Srirama

Outline

• Summarize what we have learnt

• Some more PaaS

– Aneka

– Google App Engine– Google App Engine

– Force.com

• Amazon SQS

24.05.2011 Satish Srirama 2

What is Cloud Computing?

• Computing as a utility

– Consumers pay based on their usage

• Cloud Computing characteristics

– Illusion of infinite resources

– No up-front cost

– Fine-grained billing (e.g. hourly)

• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”

Satish Srirama24.05.2011 3/30

Cloud Computing – Services - Recap

• Software as a Service – SaaS– A way to access applications

hosted on the web through your web browser

• Platform as a Service – PaaS– A pay-as-you-go model for IT

resources accessed over the

SaaS

Facebook, Flikr, Myspace.com,

Google maps API, Gmail

Level of

Abstraction

resources accessed over the Internet

• Infrastructure as a Service –IaaS– Use of commodity computers,

distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data

– Virtualization

Satish Srirama

PaaS

Google App Engine,

Force.com, Hadoop, Azure, Amazon S3, etc

IaaSAmazon EC2, Joyent

Accelerators, Nirvanix Storage

Delivery Network, etc.

24.05.2011 4

Scientific Computing Cloud (SciCloud)

• Distributed Systems Group owned private cloud infrastructure

• Eucalyptus setup

• Goal of the project• Goal of the project

– To Establishing a private cloud at universities

– To efficiently use the already existing resources of universities

– To address computationally intensive scientific, mathematical, and academic problems

• Hope it was a pleasant experience !!!

24.05.2011 5Satish Srirama

Economics of Cloud Providers –

Failures - Recap

• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter

• Caveat: User software has to adapt to failures• Caveat: User software has to adapt to failures

• Solution: Replicate data and computation– MapReduce & Distributed File System

• MapReduce = functional programming meets distributed processing on steroids – Not a new idea… dates back to the 50’s (or even 30’s)

24.05.2011 6/30 Satish Srirama

MapReduce

• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together

• The execution framework handles everything else…• Not quite…usually, programmers also specify:• Not quite…usually, programmers also specify:

partition (k’, number of partitions) → partition for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic


Hadoop Processing Model

• Create or allocate a cluster

• Put data onto the file system (HDFS)– Data is split into blocks

– Replicated and stored in the cluster

• Run your job• Run your job– Copy Map code to the allocated nodes

• Move computation to data, not data to computation

– Gather output of Map, sort and partition on key

– Run Reduce tasks

• Results are stored in the HDFS


MapReduce Examples

• Distributed Grep

• Count of URL Access Frequency

• Reverse Web-Link Graph

• Inverted Index • Inverted Index

• Distributed Sort

• MapReduce in Information Retrieval


Synchronization in Hadoop

• Approach 1: turn synchronization into an ordering problem– Sort keys into correct order of computation

– Partition key space so that each reducer gets the appropriate set of partial results

– Hold state in reducer across multiple key-value pairs to perform computationcomputation

– Illustrated by the “pairs” approach

• Approach 2: construct data structures that “bring the pieces together”– Each reducer receives all the data it needs to complete the

computation

– Illustrated by the “stripes” approach


SOME MORE PAAS


Aneka: The Cloud Application

Platform for Resource-Intensive Apps

• PaaS from Melbourne Clouds Lab

• Available as Manjrasoft product

• Lightweight Container hosting multiple servicesservices

• All programming models available from within the same container (unlike others)

• SDK containing APIs for multiple programming models and tools


Aneka - continued

• Runtime Environment for managing application execution management

• Suitable for– Development of Enterprise Cloud Applications– Development of Enterprise Cloud Applications

– Cloud enabling legacy applications

• Portability for Customer Apps:– Enterprise ↔ Public Clouds

– .NET/Win ↔ Mono/Linux


Aneka - architecture

ContainerPersistence

Task

Model

Thread

Model

Map Reduce

ModelOther

Models Security

Programming Models

Software Development Kit

ManagementStudio

Application

Foundation Services

APIsDesign Explorer

Management Kit

AdministrationPortal

SLA-NegotiationWeb Services

ManagementWeb Services

ContainerPersistence

Task

Model

Thread

Model

Map Reduce

ModelOther

Models Security

Programming Models

Software Development Kit

ManagementStudio

Application

Foundation Services

APIsDesign Explorer

Management Kit

AdministrationPortal

SLA-NegotiationWeb Services

ManagementWeb Services


Private Cloud

LAN network

Amazon

Microsoft GoogleIBM

Data Center

Hardware Profile Services

Persistence

.NET @ Windows Mono @ Linux

Security

Membership

Services

Reservation

Services

License

Services

Storage

Services

Accounting

Services

Fabric Services

Dynamic Resource Provisioning Services

Infrastructure

Physical Machines/Virtual Machines

Private Cloud

LAN network

Private Cloud

LAN network

Amazon

Microsoft GoogleIBM

Data Center

Amazon

Microsoft GoogleIBM

Data Center

Hardware Profile Services

Persistence

.NET @ Windows Mono @ Linux

Security

Membership

Services

Reservation

Services

License

Services

Storage

Services

Accounting

Services

Fabric Services

Dynamic Resource Provisioning Services

Infrastructure

Physical Machines/Virtual Machines

Aneka and Hour Glass Model

(many in one, one in many)

multiple applicationsThread Task ... MapReduce

multiple applications


TCP/IP

Internet ArchitectureAneka Architecture

multiple networksMulti-core Cluster Grid Cloud

2100 2100 2100 2100

2100 2100 2100 2100

Aneka

Aneka: components

public DumbTask: ITask

{

…

public void Execute()

{

……

}

}

for(int i=0; i<n; i++)

{

…

DumbTask task = new DumbTask();

app.SubmitExecution(task);

}Executor Executor

Aneka enterprise

Cloud

work units


}

Executor

Scheduler

Executor

Client

Agent

work units

internet

internet

Client

Agent

Aneka User Agent

Aneka Worker ServiceAneka Manager

Programming / Deployment Model

Aneka – management studio


Aneka Support for InterCloud /

Federation of Resource Pools

• XenServer Pool– Provisioning over private Cloud managed by Xen Server

• VMWare Pool

Executor

Executors

Schedulerprivate enterprise

Executor


• VMWare Pool– Provisioning over private Cloud managed by VMWare

• Amazon EC2 Pool– Provisioning over public Cloud provider: Amazon EC2

Executors/SchedulersClient Libraries

Scheduler

Public Cloud

publicly available resources

(physical and virtual)

internet

Private Cloud

private enterprise

network

VPN

(virtual resources)

Aneka cloud at UT

• We will soon be establishing Aneka cloud at

UT

– Probably in summer

– Already have the basic setup ready– Already have the basic setup ready

• Check for updates

• If anybody is interested; contact me directly

– Can be taken as an internship topic


Google AppEngine


What is AppEngine?

• Google platform to build web application on

the Cloud

• Example of PaaS

• Does one thing well: running web apps• Does one thing well: running web apps

• Request → Processing → Response

2124.05.2011 Satish Srirama

No More LAMP


Why App Engine


Application Life Cycle


Development Life cycle


Physical Deployment Diagram


APIs


End User Administration

Access Control Authentication / SSO Services

Web Services API

Workflow Engine

A B D

C

Powerful Application Service

Multi-Device Multi-LanguageMulti-

CurrencyReporting &

AnalyticsSearchEngine

28

Force.com - cloud platform for business apps


Infrastructure

Services

Network

Storage

Operating System

Database

Application

Services

Authentication

Security

Sharing

Integration

Operations

Services

Availability

Monitoring

Patch Mgmt

Upgrades

IT

Force.com Lets You Focus on

Innovation

You

InnovationApp Server

Web Server

Data Center

Disaster Recovery

Customization

Web Services

API

Multi-Language

Multi-Currency

Workflow

Analytics

Multi-Device

Messaging

Search

Backup

NOC

Sales force2924.05.2011 Satish Srirama

Create any table

Add any field

Build any relationship

Database as a Service

Build any relationship

Security & sharing

Automatic tuning

Automatic backups

Automatic upgrades


Integration as a Service


History Tracking

Workflow Rules

Logic As A Service

Declarative Logic

Formula-Based LogicFormula Fields

Procedural Logic

Formula Fields

Validation Rules

Apex Triggers

Apex Classes

Apex Web Services


User Interface as a Service

UI generated

automatically,

no technical

skills required

UI generated

by developer /

technologist

Profile /

Record Type /

Page Layout

mapping

controls UI

behavior

Full

Page Layout Visualforce

Limited

control over

look and feel,

but all UIs are

consistent

Full

control of UI

behavior

Point and click to

add Custom

Buttons and

Embedded

Mashups


LEFTOVERS


Amazon Simple Queue Service (SQS)

• A reliable, highly scalable hosted distributed queue

for storing messages as they travel between

computers.

• Amazon's web-scale messaging infrastructure as a

web service

• Platform-agnostic, allowing any computer on the

Internet to add or read messages without any

installed software or special firewall configurations


Amazon SQS Highlights

• Reliable

– Runs within Amazon's high-availability data centers.

– Messages are stored redundantly across multiple servers and data centers.

• Simple: Only Six Methods• Simple: Only Six Methods

• Scalable

• Secure: Strong access controls

• $0.01 per 10,000 messages sent ($0.000001 per message sent)

• $0.20 per GB of data transferred


Next Week

• All the best with the exams

• On 25th and 26th May 2011, we will have the

project presentations

• Examination on 31st May• Examination on 31st May

– Duration – 2 hours

– 10:15 – 12:15

– Room probably the lecture hall


References

• Aneka

http://www.manjrasoft.com/aneka_architectu

re.html

• Google App Engine • Google App Engine

http://code.google.com/appengine/

• Force.com http://www.salesforce.com/force/

• Amazon SQS http://aws.amazon.com/sqs/


Cloud Computing Lec7

Documents

Transcript of Cloud Computing Lec7