Post on 15-May-2015
Prasoon Kumar
Senior Technical Architect
Justdial Ltd
Fundamentals of Cloud Computing
IWillStudy.com Cloud Summit Ahmedabad
Introduction
Cloud Computing Industry is growing According to Gartner, worldwide cloud services revenue is on
pace to surpass $56.3 billion in 2009, a 21.3% increase from 2008 revenue of $46.4 billion, according to Gartner, Inc. The market is expected to reach $150.1 billion in 2013.
Businesses are increasing Cloud adoption "We expect a great deal of migration towards cloud computing
within the federal government in addition to the already robust private sector growth. The growth of the cloud should not outpace our ability to protect the data that goes into it..." ~ Former White House advisor Paul Kurtz, partner with Good Harbor Consulting, LLC
How can IT leaders ensure security in the cloud?
IWillStudy Cloud Summit Ahmedabad
Cloud Basics Cloud Characteristics
Service Models SaaS
IaaS
PaaS
Deployment Models Public
Private
Community
Hybrid
IWillStudy Cloud Summit Ahmedabad
Cloud Computing: Why Now?
Experience with very large datacenters
Unprecedented economies of scale
Transfer of risk
Technology factors
Pervasive broadband Internet
Maturity in Virtualization Technology
Business factors
Minimal capital expenditure
Pay-as-you-go billing model
IWillStudy Cloud Summit Ahmedabad
And in a non-Cloud view, there are inefficiencies in
addressing those issues
TIME
IT C
AP
AC
ITY
Actual Load
Allocated IT-capacities
“Waste“ of capacities
“Under-supply“ of capacities
Fixed cost of IT-capacities
Load Forecast
Barrier forinnovations
However, in a Cloud View
Actual Load
Allocated IT
capacities
Reduction of initial
investments
Reduction of “over-supply“
No “under-supply“
Possible reduction of IT-capacities
in case of reduced load
Time
IT C
AP
AC
ITY
Load Forecast
Cloud Service Models
Software as a Service
(SaaS)
Platform as a Service
(PaaS)
Infrastructure as a Service
(IaaS)
IWillStudy Cloud Summit Ahmedabad
Vendors
IaaS
Amazon Elastic Compute
Cloud (EC2)
Eucalyptus
GoGrid
RackSpace Cloud
PaaS
Force.Com
Google App Engine
Windows Azure Platform
GoGrid CloudCenter
SaaS
Google Apps
SalesForce.Com
Oracle On Demand
SQL Azure
IWillStudy Cloud Summit Ahmedabad
Natural Evolution of the Web
Source: Lew Tucker, Introduction to Cloud Computing for Enterprise UsersIWillStudy Cloud Summit Ahmedabad
“OpenStack is a collection of open source software projects that enterprises/service providers can use to setup and run their cloud compute and storage infrastructure.”
— docs.openstack.org
The OpenStack Consortium has grown rapidly in the
past year:
• NASA
• Rackspace
• Citrix
• Dell
• AMD
OpenStack services are available via Amazon’s S3
and EC2 APIs. Applications written for Amazon Web
Services will work with OpenStack.
• Intel
• Cisco
• HP
• Over 140 others
IWillStudy Cloud Summit Ahmedabad
IWillStudy Cloud Summit Ahmedabad
Introducing Windows Azure
ON-PREMISES
WEB & CLOUDS Third party cloudWeb applications
LOB ApplicationsComposite applications
Compute Storage Management ManagementRelational data Connectivity Access control
Google App Engine Based on non-proprietary technology: Java servlet API version 2.5 or Python
Easy to learn Rich tooling Extendable (can use 3rd party tools like GWT, Guice,
Spring, JSF, Struts, Grails, Restlet) Commercial support Google has proven they are good at this stuff
IWillStudy Cloud Summit Ahmedabad
IWillStudy Cloud Summit Ahmedabad
GAE building blocks Java Servlet Environment running in Java 6 sandbox
Version 2.5
Storing data
Object database != relational: denormalized -> optimized for reading
Designed for searching in very large datasets
Multiple flavours: JDO (and JPA or native), @nnotation based
Relationships: owned (1:1, 1:n) and unowned (1:n, n:1, n:n)
Entity groups
Transactions
Indexes
URL fetch
Integration with other systems through HTTP
Send and receive email
XMPP
Send and receive instant messages
Channel
Persistent connection: no need for client side polling
IWillStudy Cloud Summit Ahmedabad
GAE building blocks (continued) Blobstore
Manage large data chunks (like files, images)
Image processing
Transform, resize, change format
Memcache
Minimize datastore hits
Cron
Schedule repetitive, non-interactive jobs
Task Queues
Asynchronous, non-interactive background processing
Authentication and SSO
Google account
Open ID
Multi-tenancy
Support different user groups
Many 3rd party libraries
But not all!
IWillStudy Cloud Summit Ahmedabad
AWS Management Console
Central location to all your aws cloud apps
EC2
Beanstalk
S3
RDS
…
Create AMI signing certificate mkdir ~/.ec2
cd ~/.ec2
openssl genrsa -des3 -out pk-<group>.pem 2048
openssl rsa -in pk-<group>.pem -out pk-unencrypt-<group>.pem
openssl req -new -x509 -key pk-<group>.pem -out cert-<group>.pem -days 1095
Share all three .pem files manually with group members
Troubleshooting: If your client date is wrong your certs will not work
Upload certificate to AWS via IAM page Login at: https://283072064258.signin.aws.amazon.com/console
Account: 283072064258
Username: group** (e.g. group1, group10, group18)
Password: In email from Dr. Jin (12 digits, something like N9EzPxXGw0Gg)
Click IAM tab -> users -> select yourself (use right arrow if needed)
In bottom pane select “Security Credentials” tab and click “Manage Signing Certificates”
Click “Upload Signing Certificate”
cat ~/.ec2/cert-<group>.pem
Copy contents into „Certificate Body‟ textbox and click „OK‟
Setting up EC2 account and tools
Retrieve and unpack AWS tools wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip ec2-api-tools.zip
Create ec2 initialization script vi ec2-init.sh (you can use your preferred editor)
export JAVA_HOME=/usr
export EC2_HOME=~/ec2-api-tools-1.5.2.4
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=~/.ec2/pk-unencrypt-<group>.pem
export EC2_CERT=~/.ec2/cert-<group>.pem
source ec2-init.sh
This will need to be done every login
Alternately, put it in ~/.profile to have it done automatically on login
Test it out ec2-describe-regions
ec2-describe-images -o self -o amazon
Troubleshooting http://docs.amazonwebservices.com/AmazonEC2/gsg/2007-01-03/
Setting up EC2 account and tools
Create a new keypair (allows cluster login) ec2-add-keypair <group>-keypair | grep –v KEYPAIR > ~/.ec2/id_rsa-<group>-
keypair
chmod 600 ~/.ec2/id_rsa-<group>-keypair
Only do this once! It will create a new keypair in AWS every time you run it
Share private key file between group members, keep it private
Don’t delete other groups’ keypairs!
Everyone has access to everyone else’s keypairs from the AWS console
EC2 tab ->Network and Security -> Keypairs
Troubleshooting
http://docs.amazonwebservices.com/AmazonEC2/gsg/2007-01-03/
Setting up EC2 account and tools
Cleanup
Terminate your clusters when you’re done!They cost <personX> grant money ($1/hour for a full cluster of 9 nodes)
You can always create more later
hadoop-ec2 terminate <group>-cluster
They can also be terminated manually from the AWS->EC2 console
GAE Demo
Course Builder on Google App engine
IWillStudy Cloud Summit Ahmedabad
http://www.iwillstudy.com/group/cloudcomputing
Join and Connect!
IWillStudy Cloud Summit Ahmedabad
http://www.iwillstudy.com/group/cloudcomputing