ACCU Report from GS User Commission G. Passaleva June 15, 2011 ACCU.
Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.
-
Upload
charity-bell -
Category
Documents
-
view
215 -
download
1
Transcript of Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.
![Page 1: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/1.jpg)
Overview of Cloud Computing
Sven RosvallACCU 2011-04-16
![Page 2: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/2.jpg)
Takeaways
• Understanding what Cloud Computing is• Players in the market• Building blocks of Cloud Computing and how to use them• Can my system be migrated?• Not covering Map/Reduce here
![Page 3: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/3.jpg)
Defining Cloud Computing
Business - A method to address scalability and availability concerns for large scale applications.
Engineering - Providing services on virtual machines allocated on top of a large physical machine pool.
Practical - Find spare capacity automatically.
Big picture - Democratized distributed computing
![Page 4: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/4.jpg)
Cloud Computing Offering
Application Programming
Automated Service
Management
Multi-Tenancy
High Availability
Fast and Massive Scale Out
![Page 5: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/5.jpg)
Scaling Scenarios
Average Usage
Com
pu
te
Time
“Growing Fast”
Com
pu
te
Time
“On and Off”
UsageAverage
InactivityPeriod
Com
pu
te
Time
“Unpredictable Bursting”
Average Usage Com
pu
te
Time
“Predictable Bursting”
Average Usage
![Page 6: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/6.jpg)
• turbotax.com • taxcut.com
• hrblock.com • taxact.com
Source: Alexa
~4x normal load(Holiday shopping)
~10x normal load(Tax season)
• target.com • walmart.com
• toysrus.com • barnesandnoble.com
Jan 2009 Jan 2010 Jan 2009 Jan 2010
Source: Alexa
Scaling in Real Life
![Page 7: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/7.jpg)
Everything as a Service
Infrastructure As a Service
• Provides VMs• Good for Legacy Apps• Not great for scalability
PlatformAs a Service
• Provides APIs and Building Block Services
• Requires a migration of apps
• Great opportunity for scalability
Software As a Service
• Provides Web Apps• Multi-Tenant• Logical Separation of data• Commodity• Very little customisation• Seamless scaling
![Page 8: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/8.jpg)
Everything as a Service
Infrastructure As a Service
PlatformAs a Service
Software As a Service
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
Managed b
y
vendor
You
manage
Managed b
y
vendor
You
manage
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
Managed b
y
vendor
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
![Page 9: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/9.jpg)
Main Providers
Infrastructure As a Service
PlatformAs a Service
Software As a Service
![Page 10: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/10.jpg)
Security
Access control Access keys, encryptionWhere is my stuff? Private clouds
Security procedures
![Page 11: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/11.jpg)
Basic Building Blocks
• RESTful API’s• Simple storage– Blobs– Tables– Queues
• Computing Entities
![Page 12: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/12.jpg)
Blobs
• Unstructured data• Reference object with a URL• Basic data storage unit
![Page 13: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/13.jpg)
Tables
• Structured data• “Entity Database”, non-relational• Sparse tables• Set of key-value pairs• Reference table with a URL
![Page 14: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/14.jpg)
Queues
• Message passing between components.• Guaranteed delivery (at least once)• Reference queue with a URL
![Page 15: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/15.jpg)
IaaS
VM
VM
PaaSVM
Computing
VM
SaaS
![Page 16: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/16.jpg)
Trends
• Relational databases• Geo-failover• Network traffic management• Edge computing
![Page 17: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/17.jpg)
Content Delivery Network
• Caches static content close to users
![Page 18: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/18.jpg)
Developing for the Cloud
• Think big!– Millions of users?– Network load?– Store TB of data?
• Prepare for scale– Simplicity– Bottlenecks?– Single Point of Failure?
• Prepare for failures– Network– Crashing servers– Server updates
• Migrating?– From SOA?– From RDB?– IaaS or PaaS?
![Page 19: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/19.jpg)
Concurrency Issues in the Cloud
• Lots of simultaneous operations• Rare problems happens– Network– Crashing servers– Server updates
• No transactions
![Page 20: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/20.jpg)
Design Practices
• Design for fault tolerance– Retry– Idempotency
• Synchronisation– Optimistic Consistency
![Page 21: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/21.jpg)
Place an order
Solution: Idempotent operations
![Page 22: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/22.jpg)
Idempotent Operations
• Can be applied multiple times with same result.
Examples:• Lookup your address• Change your address• HTTP GET/PUT/DELETE (not POST)
![Page 23: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/23.jpg)
Updating
€100
Read = €100
Write = €80
-€20€80
![Page 24: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/24.jpg)
Updating
€100
Read = €100
+€50
Write = €80
Read = €100
Write = €150
-€20€150
?
![Page 25: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/25.jpg)
Optimistic Concurrency
• Get version marker (E-Tag) when reading object
• Only allow write if version marker is unchanged.
![Page 26: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/26.jpg)
Updating
€100 v1
Read = €100 v1
+€50
Write = €80 v1
Read = €100 v1
Write = €150 v1
-€20€150 v2
X
![Page 27: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/27.jpg)
Updating
Read = €150 v2
+€50
Write = €130 v2
Read = €100 v1
Write = €150 v1
-€20€150 v2€130 v3
![Page 28: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/28.jpg)
Job Control
1. A job is managed by a message on a queue.2. A worker picks up a message and executes
the job.3. If the worker crashes, the message is
returned to the queue after a timeout. Another worker picks up the message.
4. The message is only deleted from the queue when the job is complete.
![Page 29: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/29.jpg)
Demo – MiniBing
Web Crawler
Internet
Web CrawlerWeb CrawlerWeb CrawlerWeb CrawlerWeb CrawlerWeb CrawlerWeb Indexer
Web CrawlerWeb CrawlerUI FrontendIndexing queue
Web pages
Blob Storage
Crawling queue
Web index
Table storage
blob blob blob
blob blob blob
Worker RoleWeb Role
URLs
Bootstrapping the crawler
Search query Query results
![Page 30: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/30.jpg)
Resources
• http://en.wikipedia.org/wiki/Cloud_computing• http://aws.amazon.com/• http://www.microsoft.com/windowsazure/• http://code.google.com/appengine/• http://www.salesforce.com/
![Page 31: Overview of Cloud Computing Sven Rosvall ACCU 2011-04-16.](https://reader035.fdocuments.us/reader035/viewer/2022062718/56649e9d5503460f94b9edbe/html5/thumbnails/31.jpg)
Questions?