Cloud Computing Lec7 V1kodu.ut.ee/~srirama/cloud/L7_Leftovers.pdf · Outline • Summarize what we...

Basics of Cloud Computing – Lecture 7

Cloud Computing – Summary and

leftoversleftovers

Satish Srirama

Outline

• Summarize what we have learnt

• Amazon SQS

• Some more PaaS

– Google App Engine– Google App Engine

– Force.com

• AJAX

• Virtualization

24.05.2010 Satish Srirama 2

What is Cloud Computing?

• Computing as a utility

– Consumers pay based on their usage

• Cloud Computing characteristics

– Illusion of infinite resources

– No up-front cost

– Fine-grained billing (e.g. hourly)

• Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”

Satish Srirama24.05.2010 3/30

Cloud Computing – Services - Recap

• Software as a Service – SaaS– A way to access applications

hosted on the web through your web browser

• Platform as a Service – PaaS– A pay-as-you-go model for IT

resources accessed over the

SaaS

Facebook, Flikr, Myspace.com,

Google maps API, Gmail

Level of

Abstraction

resources accessed over the Internet

• Infrastructure as a Service –IaaS– Use of commodity computers,

distributed across Internet, to perform parallel processing, distributed storage, indexing and mining of data

– Virtualization

Satish Srirama

PaaS

Google App Engine,

Force.com, Hadoop, Azure, Amazon S3, etc

IaaSAmazon EC2, Joyent

Accelerators, Nirvanix Storage

Delivery Network, etc.

24.05.2010 4

Scientific Computing Cloud (SciCloud)

• Distributed Systems Group owned private cloud infrastructure

• Eucalyptus setup

• Goal of the project• Goal of the project

– To Establishing a private cloud at universities

– To efficiently use the already existing resources of universities

– To address computationally intensive scientific, mathematical, and academic problems

• Hope it was a pleasant experience !!!

24.05.2010 5Satish Srirama

Economics of Cloud Providers –

Failures - Recap

• Cloud Computing providers bring a shift from high reliability/availability servers to commodity servers– At least one failure per day in large datacenter

• Caveat: User software has to adapt to failures• Caveat: User software has to adapt to failures

• Solution: Replicate data and computation– MapReduce & Distributed File System

• MapReduce = functional programming meets distributed processing on steroids – Not a new idea… dates back to the 50’s (or even 30’s)

24.05.2010 6/30 Satish Srirama

MapReduce

• Programmers specify two functions:map (k, v) → <k’, v’>*reduce (k’, v’) → <k’, v’>*– All values with the same key are reduced together

• The execution framework handles everything else…• Not quite…usually, programmers also specify:• Not quite…usually, programmers also specify:

partition (k’, number of partitions) → partition for k’– Often a simple hash of the key, e.g., hash(k’) mod n– Divides up key space for parallel reduce operationscombine (k’, v’) → <k’, v’>*– Mini-reducers that run in memory after the map phase– Used as an optimization to reduce network traffic


MapReduce Examples

• Distributed Grep

• Count of URL Access Frequency

• Reverse Web-Link Graph

• Inverted Index • Inverted Index

• Distributed Sort

• MapReduce in Information Retrieval


Hadoop Processing Model

• Create or allocate a cluster

• Put data onto the file system– Data is split into blocks

– Replicated and stored in the cluster

• Run your job• Run your job– Copy Map code to the allocated nodes

• Move computation to data, not data to computation

– Gather output of Map, sort and partition on key

– Run Reduce tasks

• Results are stored in the HDFS


LEFTOVERS


Amazon Simple Queue Service (SQS)

• A reliable, highly scalable hosted distributed queue

for storing messages as they travel between

computers.

• Amazon's web-scale messaging infrastructure as a

web service

• Platform-agnostic, allowing any computer on the

Internet to add or read messages without any

installed software or special firewall configurations


Amazon SQS Highlights

• Reliable

– Runs within Amazon's high-availability data centers.

– Messages are stored redundantly across multiple servers and data centers.

• Simple: Only Six Methods• Simple: Only Six Methods

• Scalable

• Secure: Strong access controls

• $0.10 per 1,000 messages sent ($0.0001 per message sent)

• $0.20 per GB of data transferred


Google AppEngine


What is AppEngine?

• Google platform to build web application on

the Cloud

• Example of PaaS

• Does one thing well: running web apps• Does one thing well: running web apps

• Request → Processing → Response

1424.05.2010 Satish Srirama

No More LAMP


Why App Engine


Application Life Cycle


Development Life cycle


Physical Deployment Diagram


Developing applications with App Engine


Developing Applications

App Engine SDK

• Web Server

dev_appserver.py

• Application uploader• Application uploader

appcfg.py

• Local DataStore

implementation

• App Engine APIs


App configuration – app.yaml

application: helloworld

version: 1

runtime: python

api_version: 1 api_version: 1

handlers:

- url: /.*

script: helloworld.py


Helloworld.py


Running Application


Data Store

• A powerful non relational distributed data

storage service(object database)

• Transactional data store

• Query engine• Query engine

– GQL – SQL like language

• Use BigTable in back


Deployment and Testing

• Local testing

dev_appserver.py helloworld/

• Deploy

appcfg.py update helloworld/appcfg.py update helloworld/

• On the cloud

http://distributedseminartartu.appspot.com


APIs


End User Administration

Access Control Authentication / SSO Services

Web Services API

Workflow Engine

A B D

C

Powerful Application Service

Multi-Device Multi-LanguageMulti-

CurrencyReporting &

AnalyticsSearchEngine

28

Force.com - cloud platform for business apps


Infrastructure

Services

Network

Storage

Operating System

Database

Application

Services

Authentication

Security

Sharing

Integration

Operations

Services

Availability

Monitoring

Patch Mgmt

Upgrades

IT

Force.com Lets You Focus on

Innovation

You

InnovationApp Server

Web Server

Data Center

Disaster Recovery

Customization

Web Services

API

Multi-Language

Multi-Currency

Workflow

Analytics

Multi-Device

Messaging

Search

Backup

NOC

Sales force2924.05.2010 Satish Srirama

Create any table

Add any field

Build any relationship

Database as a Service

Build any relationship

Security & sharing

Automatic tuning

Automatic backups

Automatic upgrades


Integration as a Service


History Tracking

Workflow Rules

Logic As A Service

Declarative Logic

Formula-Based LogicFormula Fields

Procedural Logic

Formula Fields

Validation Rules

Apex Triggers

Apex Classes

Apex Web Services


User Interface as a Service

UI generated

automatically,

no technical

skills required

UI generated

by developer /

technologist

Profile /

Record Type /

Page Layout

mapping

controls UI

behavior

Full

Page Layout Visualforce

Limited

control over

look and feel,

but all UIs are

consistent

Full

control of UI

behavior

Point and click to

add Custom

Buttons and

Embedded

Mashups


Next Week

• All the best with the exams

• On 27th May 2010, we will have the project

presentations


References

• Google App Engine

http://code.google.com/appengine/

• Force.com http://www.salesforce.com/force/

• Amazon SQS http://aws.amazon.com/sqs/• Amazon SQS http://aws.amazon.com/sqs/


Virtualization

• Virtualization technologies partition hardware

and thus provide flexible and scalable

computing platformsApp App App

• Types of Virtualization

– Hardware Virtualization

– Virtual Machine Virtualization

– Application Virtualization


Hardware

OS

Hypervisor

OS OS

Virtualized Stack

Hardware Partitioning

• Physical system is

divided into multiple

partitions

• One or more CPUs per • One or more CPUs per

partition

• Full electrical isolation

between partitions

• Redundant hardware

components


Virtual Machine Virtualization

Virtual MachineVirtual Machine

Guest OSGuest OS

(Windows 2003)(Windows 2003)

ApplicationsApplications

Virtual MachineVirtual Machine Virtual MachineVirtual Machine

Guest OSGuest OS

(Windows NT)(Windows NT)

Guest OSGuest OS

(Windows 2000)(Windows 2000)

ApplicationsApplications ApplicationsApplications

V i

r t

u a

l

System HardwareSystem Hardware

Host Operating SystemHost Operating System

Virtual Machine ManagerVirtual Machine Manager

P h

y s

i c

a l

Virtualization platforms

• XEN– XEN 3.0 -- Open source

• Limited guest support

• Requires OS Source code changes

– XEN Enterprise – commercial– XEN Enterprise – commercial• Windows and Linux support

• Virtuozzo– Linux roots

– Virtualizes the OS rather than the hardware

• VMWare


Application Virtualization

• Runs as an application

• Virtualizes OS settings

– Registry– Registry

– File System

Ajax

• Asynchronous JavaScript and XML: an approach for building interactive Web applications

• Ajax refers to a number of technologies:• Ajax refers to a number of technologies:

– XHTML/CSS for presentation

– XML for data exchange (or JSON)

– XMLHttpRequest object for asynchronous communication

– JavaScript to tie everything together

From “old-school” Web applications to Ajax…


“Old-School” Web Applications

server-side systemsbrowser

1

user does something

2

browser sends

request to server

3

server generates Web

page as a response to

the request

backend

database

Inte

rfa

ce

Web

server

HTTP request

HTTP response

2

4

data is returned

in response to the

request

5

browser replaces view with

data sent from server


Characteristics

• User-driven: Things only happen when the user

does something (e.g., clicks on a link or button)

• Views defined by URLs: You can bookmark

something and come back to it; use the something and come back to it; use the

forward/backward button

• Simple user interaction model: Not that many

things you can do in browser

• Synchronous Interaction: System responses are

synchronized with user-driven events


browser

Synchronous Interactions

server-side

Time

user activity user activity user activity

1

2

4

5

server-side

server processing server processing

browser server-side systemsHTTP request

HTTP response

12

45 3

3


L

A

Linux

Apache

So what do you run on the server side?

A

M

P

MySQL

PHP/Python/Perl


From “Old-School” to Ajax

browser

Ajax intermediates between the interface and the server.

server-side systems

Inte

rfa

ce request

response

backend

database

Web

server

data

management

Ajax

“engine”

interaction management


Inside the Browser

browser

Re

nd

eri

ng

En

gin

e

HTTP request

HTML / CSS

data

other data

(e.g. images)

Inte

rfa

ce

Re

nd

eri

ng

En

gin

e

HTTP response


Enter JavaScript

browser

Re

nd

eri

ng

En

gin

e

HTTP requestJavaScript Engine

HTML / CSS

data

other data

(e.g. images)

Inte

rfa

ce

Re

nd

eri

ng

En

gin

e

HTTP response

JavaScript

code


Enter Ajax

browser

Re

nd

eri

ng

En

gin

e

HTTP requestJavaScript Engine

XMLHttpRequest

HTTP request

HTML / CSS

data

other data

(e.g. images)

Inte

rfa

ce

Re

nd

eri

ng

En

gin

e

HTTP response

XML

data

JavaScript

code


AJAX Interactions

browser

user activity

client-side processing

server-side

Time

server processing server processing


Components of an Ajax Interaction

1. A client event occurs (captured by JavaScript event handlers)

2. An XMLHttpRequest object is created and configured

3. An asynchronous request is made to the server 3. An asynchronous request is made to the server via the XMLHttpRequest object

4. Server processes request and returns data, executing a callback in the XMLHttpRequest object

5. The HTML DOM is updated based on response data


DOM

• Document Object Model: platform- and

language-independent way to represent XML

– Adopts a tree-based representation

– W3C standard, supported by modern browsers– W3C standard, supported by modern browsers

• JavaScript uses DOM to manipulate content

– To process user events

– To process server responses (via XMLHttpRequest)


Ajax: Things to watch out for!

• Hype– Best thing since sliced bread?

• Application development/maintenance cost– Brower incompatibilities

– Many different approaches and tools– Many different approaches and tools

– For many things, lack of agreed-on best practices

• Behavior is not ‘Web-like’ – Standard things often don’t work correctly (e.g., browser

‘back’ button, bookmarks)

– Usability issues for users with disabilities

• Security issues– Whole new class of cross-site scripting (XSS) exploits


Cloud Computing Lec7 V1kodu.ut.ee/~srirama/cloud/L7_Leftovers.pdf · Outline • Summarize what we...

Documents

Transcript of Cloud Computing Lec7 V1kodu.ut.ee/~srirama/cloud/L7_Leftovers.pdf · Outline • Summarize what we...