© 2012 IBM Corporation
OpenStack CE Technology Review & Demo Egan Ford IBM Distinguished Engineer [email protected]
© 2012 IBM Corporation 2
PPT’s and Videos: http://xmission.com/~egan/cloud/!
© 2012 IBM Corporation 3
Agenda
• IBM SmartCloud and OpenStack • Cloud Taxonomy • Some OpenStack Public Use Cases • What is OpenStack • OpenStack Resources • IBM Resources/Solutions for OpenStack Available Today • OpenStack (Video) Demo
© 2012 IBM Corporation 4
Evolve existing infrastructure
to Cloud
Accelerate adoption with
expert integrated systems
Immediate access to a managed
platform
Common Open Standards-based Cloud Management Services
Open architectures enable real innovation through interoperability
© 2012 IBM Corporation 5
IBM embraces & invests in open source to foster innovation
Cloud Computing
Application Servers
Service Orientation
Service Oriented Architecture
Systems of Interaction
Social Business
Open Cloud Architecture
June 1998: IBM enters into an engineering agreement with The Apache Group for development of the open-source Apache HTTP server software eventually becoming the leader of the new Application Server market
September 1999: IBM capitalizes on an untapped market trend and begins participating in the community development of Linux with a $60M annual investment
November 2001: IBM rallies 150 influential vendors and the development community around a new tools environment with a $40 Million software donation disrupting the leadership of the software development ecosystem
September 2012: IBM orchestrates the launch of The OpenStack Foundation boasting $10 million in funding and 5,600 members changing the dynamics of the Cloud ecosystem
© 2012 IBM Corporation 6
An open cloud architecture is emerging…built on open technologies
Hardware
OAuth
TOSCA
OSLC
cloudfoundry.org
© 2012 IBM Corporation 7
OpenStack is a global collaboration of developers & cloud computing technologists working to produce an ubiquitous Infrastructure as a Service (IaaS) open source cloud computing platform for public & private clouds.
The OpenStack Foundation – IBM is a driving force in it’s success
Platinum Sponsors APR 2012
Sep 2013 150
Contributors 2600 Individuals
1021 Contributors 11,800+ Individuals
Exponential growth in 1+ YR Gold Sponsors
Software Defined
Environment
© 2012 IBM Corporation 8
IBM is #2 in contributions Cloud Computing
Reference Architecture (CCRA)
Cloud Standards Customer Council
(CSCC)
LinkedData, OSLC
TOSCA
IBM’s ecosystem approach to an Open Cloud Architecture
OpenStack Open Source
Reference
IBM is working to accelerate OpenStack Foundation success
Because an open interoperable Cloud is critical for flexible cloud deployment and customer success…
IBM has 12 core contributors 12
IBMers working on OpenStack – from formation of the Foundation to Code Quality & New Function 370+
IBMers have signed the CLA
2
90
© 2011 IBM Corporation IBM CONFIDENTIAL
IBM contributions to OpenStack are wide-ranging, contributing to OpenStack success and delivering real value
OpenStack Compute Provision and manage large networks of virtual machines"• Platform integration (CCS)"• High Availability enhancements (RES)"• Resource optimization (RES)"• Live upgrade contributions (LTC)"• Enablement for P & Z Systems, DB2 (CCS, SOS)"• ESXi support (SOS w/ VMWare)"• VM group enablement in scheduler (RES)"• CPU allocation for vCPUs (RES)"
OpenStack Object Store Create petabytes of secure, reliable storage using standard HW"• Block & object storage enablement for IBM capability (RES)"• Nova blueprints (LTC)"• Cinder local storage & local instance clone (CCS)"• Efficient clone image in Cinder SVC driver for cFlex (RES)"• Nova & Cinder storage blueprints (CCS, SOS)"• Storwise/SVC driver update – support iSCSI CHAP auth (SOS, RES)"• Wsgi application interface enabling external web server (RES)"• Swift / Keystone interface for Keystone v3 API (RES)"
OpenStack NetworkingCreate petabytes of secure, reliable storage using standard HW"• Support for key emerging networking standards (RES)"• Quantum blueprints & migration from Nova (LTC)"• FibreChannel support (RES)"
OpenStack Dashboard Enables administrators and users to access & provision cloud-based resources through a self-service portal."• Globalization and crowd-sourced translation integration (SOS)"• Cross hypervisor testing and validation (CCS)"
OpenStack Shared ServicesLibraries that provide image management, authentication & security across all OpenStack projects"• Security & authentication enhancements (CCS, SOS)"• Image activation for OVF (CCS)"• Guest level metric collection (CCS)"• APIs: Enablement for key emerging standards (SOS)"• Membership services enhancements (CCS, RES)"• Glance: multiple image locations (CCS)"
General OpenStack contributions"• Drive IBM value-add capability from SCP (CCS)"• Community facing contributions – bug fixing, community building & promotion (LTC, SOS)"• QA items (LTC)"
CCS – Common Cloud Stack (STG & SWG) LTC – Linux Technology Center (STG) SOS – Standards & Open Source (SWG) RES – Research
Part
icip
atio
n K
ey
© 2011 IBM Corporation IBM CONFIDENTIAL
IBM SmartCloud solutions are moving to an OpenStack-based infrastructure layer, enabling smooth migration and upgrade
Related Standards & Organizations
SmartCloud Orchestrator Orchestrate Services across multiple environments and domains
CIMI & OVF
TOSCA
CCRA OSLC
§ Simple 3 tier structure, with increased Client Value at each tier § Using open, common, standards based architecture
providing choice, flexibility, interoperability, portability § Clean upgrade paths with progression to fully integrated and
factory optimized PureApplication System § Significant customer benefits above and beyond base
OpenStack
SmartCloud Provisioning Automate Optimized
Workloads
SmartCloud Entry Automate IT Delivery
SmartCloud Provisioning Automate Optimized
Workloads
SmartCloud Entry Automate IT Delivery
Customer integrated hardware
PureFlex System PureApplication
System
Automate Optimized Workloads
Key
Common Cloud Stack
Factory Integrated
Bundle Option
PureApp Server
© 2012 IBM Corporation 11
Cloud Taxonomy
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
© 2012 IBM Corporation 12
Cloud Value Proposition and Positioning
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
© 2012 IBM Corporation 13
How You (Provider) Build These Clouds
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
© 2012 IBM Corporation 14
What You (Consumer) Get with These Clouds:
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
© 2012 IBM Corporation 15
Policy-based Clouds and Design-for-fail Clouds are purpose optimized Infrastructure Management solutions
Policy-based Clouds
• Purpose optimized for longer-lived virtual machines managed by Server Administrator
• Centralizes enterprise server virtualization administration tasks
• High degree of flexibility designed to accommodate virtualization all workloads
• Significant focus on managing availability and QoS for long-lived workloads with level of isolation
• Characteristics derived from exploiting enterprise class hardware
• Legacy applications
Design-for-fail Clouds
• Purpose optimized for shorter-term virtual machines managed via end-user or automated process
• Decentralized control, embraces eventual consistency, focus on making “good enough” decisions
• High degree of standardization • Significant focus on ensuring availability
of control plane • Characteristics driven by software • New applications
© 2012 IBM Corporation 16
Some OpenStack Public Use Cases
• Internap • http://www.internap.com/press-release/internap-announces-world%E2%80%99s-first-
commercially-available-openstack-cloud-compute-service/ • Rackspace Cloud Servers, Powered by OpenStack
• http://www.rackspace.com/blog/rackspace-cloud-servers-powered-by-openstack-beta/ • Deutsche Telekom
• http://www.telekom.com/media/media-kits/104982 • AT&T
• http://arstechnica.com/business/news/2012/01/att-joins-openstack-as-it-launches-cloud-for-developers.ars
• MercadoLibre • http://openstack.org/user-stories/mercadolibre-inc/mercadolibre-s-bid-for-cloud-
automation/ • NeCTAR
• http://nectar.org.au/ • San Diego Supercomputing Center
• http://openstack.org/user-stories/sdsc/
© 2012 IBM Corporation 17
OpenStack design tenets focus on delivering essential infrastructure on an available, scalable, elastic control plane
Sources: http://www.openstack.org/downloads/openstack-compute-datasheet.pdf http://wiki.openstack.org/BasicDesignTenets
Basic Design Tenets
1) Scalability and elasticity are our main goals
2) Any feature that limits our main goals must be optional
3) Everything should be asynchronous. If you can't do something asynchronously, see #2
4) All required components must be horizontally scalable
5) Always use shared nothing architecture (SN) or sharding. If you can't Share nothing/shard, see #2
6) Distribute everything. Especially logic. Move logic to where state naturally exists.
7) Accept eventual consistency and use it where it is appropriate.
8) Test everything. We require tests with submitted code. (We will help you if you need it)
OpenStack Leadership's vision statement
“essential Infrastructure, support platform”
© 2012 IBM Corporation 18
OpenStack
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
© 2012 IBM Corporation 19
OpenStack is comprised of seven core projects that form a complete IaaS solution
Compute (Nova) Storage (Cinder) Network (Quantum) Provision and manage virtual resources Dashboard (Horizon) Self-service portal Image (Glance) Catalog and manage server images Identity (Keystone) Unified authentication, integrates with existing systems Object Storage (Swift) petabytes of secure, reliable object storage
IaaS
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
IaaS
© 2012 IBM Corporation 20
Compute delivers a fully featured, redundant, and scalable cloud computing platform
Architecture
Sources: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/ http://openstack.org/projects/compute/
Key Capabilities:
• Manage virtualized server resources • CPU/Memory/Disk/Network Interfaces
• API with rate limiting and authentication
• Distributed and asynchronous architecture • Massively scalable and highly available system
• Live guest migration • Move running guests between physical hosts
• Live VM management (Instance) • Run, reboot, suspend, resize, terminate instances
• Security Groups
• Role Based Access Control (RBAC) • Ensure security by user, role and project
• Projects & Quotas
• VNC Proxy through web browser
© 2012 IBM Corporation 21
Compute management stack control plane is built on queue and database
Key Capabilities: • Responsible for providing communications hub and
managing data persistence
• RabbitMQ is default queue, MySQL DB • Documented HA methods • ZeroMQ implementation available to decentralize
queue
• Single “cell” (1 Queue, 1 Database) typically scales from 500 – 1000 physical machines
• Cells can be rolled up to support larger deployments
• Communications route through queue • API requests are validated and placed on queue • Workers listen to queues based on role or role +
hostname • Responses are dispatched back through queue
© 2012 IBM Corporation 22
nova-compute manages individual hypervisors and compute nodes
Key Capabilities: • Responsible for managing all interactions with individual
endpoints providing compute resource, e.g. -- Attach iSCSI volume to phsyical host, map to guest as additional HDD
• Implementations direct to native hypervisor APIs – Avoids abstraction layers that bring least common
denomination support – Enables easier exploitation of hypervisor
differentiators
• Service instance runs on every physical compute node, helps to minimize failure domain
• Support for security groups that define firewall rules
• Support for – KVM – LXC – VMware ESX/ESXi (4.1 update 1) – Xen (XenServer 5.5, Xen Cloud Platform) – Hyper V
© 2012 IBM Corporation 23
nova-scheduler allocates virtual resources to physical hardware
Key Capabilities: • Determines which physical hardware to allocate to a
virtual resource
• Default scheduler uses a series of filters to reduce set of applicable hosts and uses costing functions to provide Weight
• Not a focus point for OpenStack – Default implementation finds first fit – Shorter the workload lifespan, less critical the
placement decision
• If default does not work, often deployers have specific requirements and develop custom
© 2012 IBM Corporation 24
nova-api supports multiple API implementations and is the entry point into the cloud
Key Capabilities: • APIs supported
– OpenStack Compute API (REST-based) – Similar to RackSpace APIs
– EC2 API (subset) – Can be excluded
– Admin API (nova-manage) • Robust extensions mechanism to add new capabilities
© 2012 IBM Corporation 25
Network automates management of networks and attachments (network connectivity as a service)
Key Capabilities:
• Responsible for managing networks, ports, and attachments on infrastructure for virtual resources
• Create/delete tenant-specific L2 networks
• L3 support (Floating IPs, DHCP, routing)
• Moving to L4 and above in Grizzly
• Attach / Detach host to network
• Similar to dynamic VLAN support
• Support for • Open vSwitch • OpenFlow (NEC & Floodlight controllers) • Cisco Nexus • Niciria
Architecture
© 2012 IBM Corporation 26
Cinder manages block-based storage, enables persistent storage
Key Capabilities: • Responsible for managing lifecycle of volumes and
exposing for attachment
• Structure is a copy of Compute (Nova), sharing same characteristics and structure in API server, scheduler, etc.
• Enables additional attached persistent block storage to virtual machines
• Support for booting virtual machines from nova-volume backed storage
• Allows multiple volumes to be attached per virtual machine
• Supports following – ISCSI – RADOS block devices (e.g. Ceph distributed file
system) – Sheepdog – Zadara
Architecture
© 2012 IBM Corporation 27
Identity service offers unified, project-wide identity, token, service catalog, and policy service designed to integrate with existing systems
Key Capabilities: • Identity service provides auth credential validation and
data about Users, Tenants and Roles
• Token service validates and manages tokens used to authenticate requests after initial credential verification
• Catalog service provides an endpoint registry used for endpoint discovery.
• Policy service provides a rule-based authorization engine and the associated rule management interface.
• Each service configured to serve data from pluggable backend
– Key-Value, SQL, PAM, LDAP, PAM, Templates
• REST-based APIs
© 2012 IBM Corporation 28
Image service provides basic discovery, registration, and delivery services for virtual disk images
Key Capabilities: • Think Image Registry, not Image Repository
• REST-based APIs
• Query for information on public and private disk images
• Register new disk images
• Disk images can be stored in and delivered from a variety
of stores (e.g. SoNFS, Swift)
• Supported formats – Raw – Machine (a.k.a. AMI) – VHD (Hyper-V) – VDI (VirtualBox) – qcow2 (Qemu/KVM) – VMDK (VMWare) – OVF (VMWare, others) References
http://openstack.org/projects/image-service/
© 2012 IBM Corporation 29
Dashboard enables administrators and users to access and provision cloud-based resources through a self-service portal
Key Capabilities: • Thin wrapper over APIs, no local state
• Registration pattern for applications to hook into
• Ships with three central dashboards, a “User
Dashboard”, a “System Dashboard”, and a “Settings
• Out-of-the-box support for all core OpenStack projects • Nova, Glace, Switch, Quantum
• Anyone can add a new component as a “first-class
citizen”. • Follow design and style guide.
• Visual and interaction paradigms are maintained
throughout.
• Console Access References http://horizon.openstack.org/intro.html
© 2012 IBM Corporation 30
OpenStack Resources
• Forums • http://forums.openstack.org/
• Wiki • http://wiki.openstack.org/
• Documentation • http://docs.openstack.org/
• Mailing Lists • http://wiki.openstack.org/MailingLists
• OpenStack Project Management • https://launchpad.net/openstack
• Blogs • http://planet.openstack.org
• Real-time chat room • #openstack and #openstack-dev on irc://freenode.net (443 users currently logged in)
• Rackspace Reference Architectures • http://www.referencearchitecture.org/
• Easy Install • http://www.hastexo.com/resources/docs/installing-openstack-grizzly-20131-ubuntu-1204-precise-
pangolin
© 2012 IBM Corporation 31
IBM Resources/Solutions for OpenStack CE Available Today
• developerWorks • Google: openstack IBM developerworks
• xCAT (FOSS) for 0-day deployment
• xCAT OpenStack Paper (CATStack) • Automated qcow2 image creation for Glance • HW control • Bare-metal discovery and bring up
• Firmware, Base OS, etc…
• IBM Intelligent Cluster Solutions (see Matt Ziegler's PPT) • Preconfigured Switches • Rack and stacked and ready to go • Lab Services for 0-day
© 2012 IBM Corporation 32
IBM Resources/Solutions for OpenStack Available Today
• All IBM System Software and Tools can coexist with OpenStack. • Director, ASU, lflash, etc…
• SoNAS for shared file (NFS, SMB)
• XIV, v7000 for block storage (Cinder) • iDPX for scale-out Nova Compute and Swift
• BNT switches for OpenFlow and Quantum
• GPFS for iSCSI/block (Cinder) or file.
© 2012 IBM Corporation 33
OpenStack Demo Setup
10.0.9.10! 10.0.9.11! 10.0.9.12! 10.0.9.13! 10.0.9.X!
172.20.249.10! 172.20.249.11! 172.20.249.12! 172.20.249.13! 172.20.249.X!
os-essex0! os-essex1! os-essex2! os-essex3! os-essexX!
Control Nodes Compute Nodes
Private Networks: eth0: 172.20.249/24 vm: 172.20.250/24!
Public Networks: eth1: 10.0.9.0/25 vm: 10.0.9.128/25 !
compute!network!
compute!network!
compute!network!
compute!network!scheduler!volume!console!glance!api!
compute!network!scheduler!volume!console!glance!api!
Scale Out HA Active/Passive
VM
VM
VM
VM
VM Firewall
© 2012 IBM Corporation 34
PPT’s and Videos: http://xmission.com/~egan/cloud/!
Top Related