1

Cloud Foundry and OpenStack – Marriage Made in Heaven !

Apps

Animesh Singh, Egle Sigler, Jason Anderson@animeshsingh, @eglute, @andersonljason

CF Summit, San FranciscoJune 9-11

2

What Is OpenStack

• Open Source Project for with a very large community• Open source cloud computing platform for public and private clouds• IaaS for managing compute, storage, and networking resource• Standard and massively scalable cloud operating system

3

Who Is OpenStack

• Founded by Rackspace and NASA in 2010• Open Source global community of developers, testers, operators, and users

4

OpenStack Community

76K Commits

2000 Developers

38K Tickets

55K Mail Messages

… and counting!

http://activity.openstack.org/dash/releases/index.html?data_dir=data/whole_project

5

OpenStack Architecture

Collection of well integrated projects:• Compute (Nova)• Networking (Neutron)• Object Storage (Swift)• Block Storage (Cinder)• Identity(Keystone) • Image Service (Glance)• Dashboard (Horizon)…and others

6

OpenStack Compute (Nova)

Compute is a cloud computing fabric controller, the main part of an IaaS system.

• Component based architecture: Quickly add new behaviors• Highly available: Scale to very serious workloads• Fault-Tolerant: Isolated processes avoid cascading failures• Recoverable: Failures should be easy to diagnose, debug, and rectify• Open Standards: Be a reference implementation for a community-driven api• API Compatibility: Nova strives to provide API-compatible with popular systems like

Amazon EC2

7

OpenStack Networking (Neutron)

Network connectivity as a service, a system for managing networks and IP addresses.

Advantages of software defined networking. Create and manage:• Networks and subnets• Routers• Load balancers• Gateways• Floating IPs

8

OpenStack Block Storage (Cinder)

OpenStack Block Storage (Cinder) provides persistent block-level storage devices for use with OpenStack compute instances.

• Fully integrated with Compute and Dashboard components• Allows users to manage their own storage needs• Appropriate for performance sensitive scenarios:

• database storage • expandable file systems • providing a server with access to raw block level storage

• Snapshot management provides for volume/data backup and restoration

9

OpenStack DashboardHorizon is the canonical implementation of OpenStack’s Dashboard, which provides a web based user interface to OpenStack services including Nova, Swift, Keystone, etc.

10

Along with Rackspace, IBM is also working to accelerate OpenStack Foundation success…

Contributions across all releases

Contributions to the latest Icehouse release

http://activity.openstack.org/dash/releases/index.html

11

Why Cloud Foundry?

Meets Developer’s NeedsFocus on app development,

not provisioning VMs, databases, messaging

servers, etcAgile development model

Deploy and scale in seconds

Open Cloud PlatformThere is an increasing appetite for cloud-based mobile, social and analytics applications from line-of-business executives -

drives the need for a more open cloud development platform

Compelling Community Cloud Foundry has a compelling

community and emerging ecosystem as well as a mature set

of capabilities and robustness

Platinum Founding Sponsors1,274 711k

LINES OF CODETOTAL CONTRIBUTORS

12

IBM BlueMix (powered by Cloud Foundry)http://bluemix.net

IBM initiative to develop a Platform as a Services offering

IBM and partner cloud services

Integrated DevOps with both Browser and Eclipse-based tools

Services

Lifecycle Management

(JazzHub)

ApplicationRuntime

Runtimes &Frameworks

Middleware Application Operational Mobile ExternalData

node java ruby WorklightWebSphereLiberty

Web IDE(Eclipse Orion)

Eclipse IDEApplication

CompositionEnvironment

Create & Manage Services

Test/Run Test/Run

ExploreServices

ExploreServices

IBM Bluemix ™Check In Code Check In Code

Come, experiment with our open beta at http://bluemix.net

13

Integrate !

14

PaaS

Cloud Foundry – OpenStack integration

Cloud

Integration

IaaS

UAA

Router

DEA Pool

Service Gateway Apps

Service Connector

Health Manager

Messaging

Cloud Controller

Build Packs

Service Nodes

Cloud Foundry BOSH

Cloud Provider Interface

• Static / floating ips• Persistent disks• Custom VM

Configurations• Specialized Security

groups• Outbound Internet

connectivity

15

BOSH deployment process

Deployment Manifest• Release

name/version• # VMs, job params• Stemcells to use

Stemcell• Base OS• BOSH agent

Release• Name• Software packages• Config templates• Scripts

BOSH

DeployedCloud Foundry

Virtual Machine• Configuration• Software Packages

Virtual Machine• Configuration• Software Packages

Virtual Machine• Configuration• Software Packages

Virtual Machine• Configuration• Software packages

16

Automate (Leverage the power of community) !

17

Leverage the power of Open Source Community

• Open technologies give us a tremendous amount of power and flexibility to integrate them together seamlessly

• Lets see some of the examples of what we can do with the power of Open technologies to go from bare metal hardware to a deployed Cloud Foundry ready to push apps

Open Source for Cloud Foundry and OpenStack Automation!

18

OpenStack Installation:

• Leverage the open source Chef Cloud infrastructure Automation framework• Requires information about hardware, network environment and software repositories.• Automate OpenStack installation in approximately 45 minutes

Chef for OpenStack Install Automation

Setup Compute

Setup Network

Setup Controller

Setup Storage

19

OpenStack Discovery:

• Leverage the open source Fog gem to discover OpenStack artifacts in an automated manner • Require OpenStack credentials and discover OpenStack compute and network information.

Fog for OpenStack Discovery Automation

Discover VM Configuration Sizes

Discover NetworkSubnets

Discover Network Security Rules

Discover DHCP , DNS Gateway and floating IPs

Discover Security Credentials

20

Fog for OpenStack Setup Automation

Cloud Foundry Pre-req setup on OpenStack:

• Leverage the open source Fog gem to setup Cloud Foundry requirements in an automated manner • Setup according to best practices and guidelines – still giving users the flexibility to change if desired

Create Security Credentials

Create VM configs for Router, DEAs, Cloud Controller, Service Nodes

Create network Security Rules

Setup tenant quota

21

RUBY

BOSH and Ruby for Cloud Foundry Deployment Automation

Cloud Foundry Deployment Automation

• Automate base OS image or Stemcell modification • Automate Cloud Foundry deployment manifest file genration using Ruby ERB templates• Automate upload of Cloud Foundry core release, services and runtime frameworks, followed by Cloud

Foundry deployment

Stemcell Creation/Modification and Upload

Generate BOSH and Cloud Foundry Manifest

Upload Cloud Foundry core, Services and runtime

Deploy Cloud Foundry

Deploy Microbosh

BOSH

22

Automate end to end deployments in under 3 hours !

• Previous example signify the power of Open source community• Leveraging tools like Chef, Fog, BOSH, Ruby ERB from the community can help create powerful

automation which can be repeatedly and consistently replicated in under 3 hours !

Open way is the only way!

BOSH

RUBY

23

Scale!

24

Scaling Number of Apps on Cloud Foundry

Sample sizing for 1000 small applications

Total Resources: 160 VCPU, 500GB Memory, 1.5TB of VM disk

25

Scaling OpenStack

Optimize Internal Communication• Configure OpenStack for scaled concurrency

Optimize Performance• Configure OpenStack scheduler to evenly distribute load

Setup Highly Available Architecture for PaaS workloads

Load Balancer Nodes

DataNodes

3x

Compute Nodes

15xStorage

Nodes

2x

Controller Nodes

2x 6x

26

Scaling Cloud Foundry / BOSH

Optimize Internal Communication• Configure messaging bus for VM communication

Optimized routing and bandwidth allocation• Isolate Cloud Foundry components using multiple networks

Maintain Cloud Foundry’s Highly Available Architecture

Service Gateways

10xRouters

3x

Service Nodes

15x

DEAs20x

Cloud Controllers

2x

27

Summary

Why Cloud Foundry and OpenStack are a great fit ?

• 100% Open PaaS and IaaS solutions (no vendor lock-ins) with a growing community of contributors and sponsors on both sides

• Power of Open Source community can be leveraged to automate the deployment and lifecycle management of Cloud Foundry on OpenStack

• OpenStack meets Cloud Foundry integration requirements, and is totally configurable and adaptable to handle the scale of a PaaS solution like Cloud Foundry

Bottom Line: Its indeed a match made in Heaven !

28

What next? Join us at our Silicon Valley Meetups

http://www.meetup.com/BlueMix/

http://www.meetup.com/CloudFoundry/

29

References

Find the slides presented today here:

http://www.slideshare.net/AnimeshSingh/cloud-foundry-and-openstack-marriage-made-in-heaven-35717536For more technical details, refer to the slides and video herehttp://www.slideshare.net/AnimeshSingh/optimizing-cloud-foundry-and-openstack-for-large-scale-deploymentshttps://www.openstack.org/assets/presentation-media/A-Practical-Approach-to-HA-Final.pdfhttps://www.youtube.com/watch?v=jCwtV9n_ak4https://www.youtube.com/watch?v=FsrCGkBo4Vg