Mirantis OpenStack-DC-Meetup 17 Sept 2014

Post on 18-Dec-2014

833 views 1 download

description

Agenda: ------------------------------------------------------------------ OpenStack 101: a Quick introduction to OpenStack & how it operates Paul Roberts, Principal Solutions Architect at Mirantis Abstract: Are you new to OpenStack? Are you looking to get a quick introduction to OpenStack and how it operates - then our session is a do not miss event! Mirantis will do a walk thru of OpenStack for those with little to no experience with OpenStack. Join us if you want to understand the purpose of OpenStack and its ecosystem, as well as if you want to learn more about the OpenStack architecture. Bio: Paul Roberts, lead speaker, has spent the last decade engineering and implementing large scale infrastructure and security architectures for organizations of all sizes - ranging from startup to Fortune 500. In the past, he was instrumental in architecting Carpathia Hosting's federal and commercial cloud offerings, while also playing a key role in the on–boarding of customer's applications. Today, Paul is a Principal Solutions Architect at Mirantis helping customers navigate through the cloud ecosystem by designing and architecting various OpenStack powered initiatives.

Transcript of Mirantis OpenStack-DC-Meetup 17 Sept 2014

©  MIRANTIS  2012   PAGE  1  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2012   CONFIDENTIAL  MIRANTIS  ©  MIRANTIS  2013  

OpenStack Overview

Paul Roberts

Principal Solutions Architect, Mirantis

©  MIRANTIS  2012   PAGE  2  CONFIDENTIAL  MIRANTIS  

●  Network security startup

‑  Acquired by MCI in 2005 ●  Sun Microsystems Alumni

‑  Hosted many internal services such as Shared Shell

●  Carpathia Hosting

‑  Cloud Architect with >48PB under management ●  Coraid

‑  Principal Architect enabling customers to simplify their complex storage architectures

●  Mirantis

‑  Helping customers design fully operationalized and automated clouds

Who am I?

©  MIRANTIS  2012   PAGE  3  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Meetup Goals

• Understand current OpenStack trends • Understand OpenStack purpose and use cases • Understand the OpenStack ecosystem

•  Definition •  History •  Programs (previously called Projects)

• Understand OpenStack architecture •  Logical architecture •  Provision virtual machine (VM) request flow •  Components details

©  MIRANTIS  2012   PAGE  4  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

What is Cloud?

“Cloud  compu?ng  has  transformed  the  way  storage,  networking,  and  compute  services  are  delivered.”  

©  MIRANTIS  2012   PAGE  5  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Traditional Public Cloud is Not Cheap

“The  public  cloud  is  phenomenal  if  you  need  it’s  elas?city,  but  if  you  don’t  –  if  you  do  a  consistent  amount  of  workload  –  it’s  far,  far  beUer  to  go  in-­‐house.”  –  Eric  Frenkiel,  MemSQL/Wired  

“[Things]  that  need  really  high  performance,  in  terms  of  [input  and  output]  and  reading  and  wri?ng  to  memory  really  belong  on  bare-­‐metal  servers  or  private  setups.”  –  John  Engates,  CTO  Rackspace/Wired  

“Versus  what  we’d  get  on  the  cloud,  [private  hos?ng  is]  somewhere  between  70  and  100  ?mes  cheaper.”  –  John  Hall,  CTO  Tradesy/Wired  

©  MIRANTIS  2012   PAGE  6  CONFIDENTIAL  MIRANTIS  

OpenStack Mindshare

©  MIRANTIS  2012   PAGE  7  CONFIDENTIAL  MIRANTIS  

Cloud Job Trends

©  MIRANTIS  2012   PAGE  8  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

However, AWS still run-away winner

©  MIRANTIS  2012   PAGE  9  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Oh, Docker.

©  MIRANTIS  2012   PAGE  10  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

In the end Developers Win.

©  MIRANTIS  2012   PAGE  11  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

What is OpenStack?

As described by Wikipedia:

“OpenStack is a cloud computing project aimed at providing an

infrastructure as a service (IaaS).”

©  MIRANTIS  2012   PAGE  12  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

What is OpenStack?

As described by the OpenStack Foundation:

“Aims to produce the ubiquitous Open Source Cloud Computing platform that

will meet the needs of public and private clouds regardless of size, by

being simple to implement and massively scalable.”

©  MIRANTIS  2012   PAGE  13  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Cloud Exposed Capabilities (SPI Model)

Data  Center  (Hardware,  Servers,  Networking)  

•  Software as a Service (SaaS): •  browser or thin client

access

•  Platform as a Service (PaaS): •  remote login, to install

applications

•  Infrastructure as a Service (IaaS): •  Provision CPU, RAM, VM

©  MIRANTIS  2012   PAGE  14  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

OpenStack Capabilities

• Virtual machines (VMs) on demand •  provisioning •  snapshotting

• Networks

• Storage for VMs and arbitrary files

• Multi-tenancy •  quotas for different projects, users •  user can be associated with multiple projects

©  MIRANTIS  2012   PAGE  15  CONFIDENTIAL  MIRANTIS  

OpenStack History

Date   Rel   Programs   Type   Note  

Jul  2010   N/A       PoC   *  Rackspace  Hos?ng  &  NASA  joint  launch  

Oct  2010   Aus4n   Nova,  Swih   PoC      Feb  2011   Bexar   Nova,  Glance,  Swih   PoC      

Apr  2011   Cactus   Nova,  Glance,  Swih   PoC   **  6  month  development  cycle  starts    

Sep  2011   Diablo   Nova,  Glance,  Swih   Prod   1st  produc?on  release  (Cactus)  at  Internap  (10/27)  

Apr  2012   Essex   Nova,  Glance,  Swih,  Horizon,  Keystone   Prod   Common  web  UI  and  shared  authen?ca?on  mechanism  added    

Sep  2012   Folsom   Nova,  Glance,  Swih,  Horizon,  Keystone,  Quantum,  Cinder   Prod   OpenStack  Founda?on  

Established  

Apr  2013   Grizzly   Nova,  Glance,  Swih,  Horizon,  Keystone,  Quantum,  Cinder   Prod   Ceilometer  and  Heat  incuba?on  

projects  added  

Oct  2013   Havana   Nova,  Glance,  Swih,  Horizon,  Keystone,  Neutron,  Cinder,  Heat,  Ceilometer   Prod   Quantum  is  renamed  to  Neutron  

Apr  2014   Icehouse  Nova,  Glance,  Swih,  Horizon,  Keystone,  Neutron,  Cinder,  Heat,  Ceilometer,  Trove,  Savanna,  Ironic,  Marconi  

Prod   Limited  upgrade  path  from  Grizzly  is  available  

*  Pre-­‐July  2010  is  predicated  by  Rackspace  Cloud  Files  project  (Swih),  NASA  Nebula  project  (Nova)  

©  MIRANTIS  2012   PAGE  16  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

OpenStack Integrated Programs

•  Compute (Nova) •  Networking (Neutron) •  Object Store (Swift) •  Block Storage (Cinder) •  Image Service (Glance) •  Identity (Keystone) •  Dashboard (Horizon) •  Telemetry Service (Ceilometer) •  Orchestration Service (Heat) •  Database Service (Trove)

Core

Shared Services

Storage

©  MIRANTIS  2012   PAGE  17  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

OpenStack Incubation Programs

• Data Processing (Sahara)

• Queue Service (Marconi)

• Bare Metal (Ironic)

©  MIRANTIS  2012   PAGE  18  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Each OpenStack Program

•  Is also a “top-level” OpenStack component • Has an elected “Project Technical Lead” (PTL) • Has separate developers and design teams • Has a well defined public API

•  With the exception of Horizon, which is the Web GUI, all other projects have a RESTfull (JSON/HTTP) API

• Common generic API/Infrastructure (Oslo) • Has a separate database and isolated persistent

layer

©  MIRANTIS  2012   PAGE  19  CONFIDENTIAL  MIRANTIS  

OpenStack Architecture: Begining (Cactus)

©  MIRANTIS  2012   PAGE  20  CONFIDENTIAL  MIRANTIS  

OpenStack Architecture: 2 years later

©  MIRANTIS  2012   PAGE  21  CONFIDENTIAL  MIRANTIS  

Communication Types HTTP  AMQP  SQL  3rd-­‐party  

UI:  Horizon  or  CLI  

Heat  Heat  API  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Every  OpenStack  service  exposes  access  to  res6ul  API  via  HTTP  

Each  ac>on  treated  as  distributed  transac>on,  state  built  as  MQ  messages  

Each  service  updates  it’s  own  DB  with  state  informa>on  as  ac>ons  are  performed   Direct  access  calls,  ex.  Plugins,  

NetApp,  Nicira,  etc.  

©  MIRANTIS  2012   PAGE  22  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Part 1 Recap

• OpenStack – open source software for building IaaS

• OpenStack release cycle is every 6 months

• OpenStack is an umbrella over multiple independent programs (components)

• All OpenStack components talk RESTful API

• Most OpenStack components have dedicated DB (SQL) and MQ (QP), some talk to 3rd party components using their native APIs

©  MIRANTIS  2012   PAGE  23  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Use case: Provision VM

• Most common and complex process

•  Interacts with most of OpenStack components

©  MIRANTIS  2012   PAGE  24  CONFIDENTIAL  MIRANTIS  

Initial State Assumes  Project  is  created,  provisioning  quota  is  available,  user  has  an  access  to  Horizon/CLI  Cloud  Operator,  DevOp,  etc.  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

©  MIRANTIS  2012   PAGE  25  CONFIDENTIAL  MIRANTIS  

Step 1: Request VM Provisioning via UI/CLI

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Cloud  Operator,  DevOp,  etc.  

User  logs  in  to  UI  Specifies  VM    params:  name,  flavor,  keys,  etc.  and  hits  "Create"  buWon  

©  MIRANTIS  2012   PAGE  26  CONFIDENTIAL  MIRANTIS  

Step 2: Validate Auth Data Horizon  sends  HTTP  request  to  Keystone.  Auth  info  is  specified  in  HTTP  headers.  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

©  MIRANTIS  2012   PAGE  27  CONFIDENTIAL  MIRANTIS  

Step 2: Validate Auth Data - Success Keystone  sends  temporary  token  back  to  Horizon  via  HTTP.  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

©  MIRANTIS  2012   PAGE  28  CONFIDENTIAL  MIRANTIS  

Step 3: Send API Request to Nova API

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Horizon  sends  POST  request  to  Nova  API  (signed  with  given  token).  

©  MIRANTIS  2012   PAGE  29  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Auth Token Usage Neutron  

©  MIRANTIS  2012   PAGE  30  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Keystone Architecture

OpenStack  Services  

Catalog  Backend  

Token  Backend  

Policy  Backend  

Assignments  Backend  

Iden?ty  Backend  

Creden?als  Backend  

Rule  management  interface  and  rule-­‐based  authoriza>on

Contains  temporary  tokens

Contains  endpoint  registry

Contains  users  and  groups

Contains  creden>als,  e.g.  EC2  tokens

Contains  domains,  projects,  roles  and  role  assignments  

Deploys  with    its  own  DB  but  can  also  be  subs>tuted  with  LDAP  or  other  EAS  Keystone API

©  MIRANTIS  2012   PAGE  31  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova API Characteristics

• Exposes REST API via HTTP.

• Provides system for managing multiple APIs on different sub-domains. •  EC2-compatible—starting to be deprecated •  Compute API—all innovation happens here

• The only "allowed" way to interact with Nova.

• Stateless—HA-ready.

©  MIRANTIS  2012   PAGE  32  CONFIDENTIAL  MIRANTIS  

Step 4: Validate API Token

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  API  sends  HTTP  request  to  validate  API  token  to  Keystone.  

©  MIRANTIS  2012   PAGE  33  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova Database

•  In theory can be any relational database •  Most of the deployments are done with MySQL or

PostgreSQL

• Nova API talks to database via SQLAlchemy (python ORM (Object Relational Mapper))

• Database HA should be done via external tools: •  Galera •  Multi-Master replication Model for MySQL (MMM)

©  MIRANTIS  2012   PAGE  34  CONFIDENTIAL  MIRANTIS  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Step 6a: Publish Provisioning Request

Nova  API  makes  rpc.cast  to  Scheduler.  It  publishes  a  short  message  to  scheduler  queue  with  VM  info.  

Request  has  been  validated,  but  no  ac>on  has  been  taken  yet,  i.e.  which  host,  IP  address,  etc.  

©  MIRANTIS  2012   PAGE  35  CONFIDENTIAL  MIRANTIS  

Step 7: Pick up Provisioning Request

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Scheduler  picks  up  the  message  from  MQ.  

©  MIRANTIS  2012   PAGE  36  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova Scheduler

“Nova Scheduler is a daemon, which

determines, on which compute host the request should run.”

•  Only provisioning time component—not like VMware’s Distributed Resource Scheduler (DRS)

•  Typically co-located with the Cloud Controller

©  MIRANTIS  2012   PAGE  37  CONFIDENTIAL  MIRANTIS  

Step 8a: Schedule Provisioning

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Scheduler  fetches  informa>on  about  the  whole  cluster  from  database,  filters,  selects  compute  node  and  updates  DB  with  its  ID  

©  MIRANTIS  2012   PAGE  38  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova Scheduler: Filtering

Affinity,  An>-­‐affinity,  etc.  

Eliminate  inapplicable  hosts  

©  MIRANTIS  2012   PAGE  39  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova Scheduler: Examples

•  Based on Host statically configured properties •  SimpleCIDRAffinityFilter •  AvailabilityZoneFilter

•  Based on already running individual VMs •  SameHostFilter •  DifferentHostFilter

•  Collocate/Distribute group of VMs •  GroupAffinityFilter, •  GroupAntiAffinityFilter

•  Based on Host resources left •  CoreFilter,

AggregateCoreFilter •  RamFilter,

AggregateRAMFilter •  DiskFilter

•  Based on Host load •  IoOpsFilter •  NumInstancesFilter

•  Based on image used •  ImagePropertiesFilter

•  Write your own

©  MIRANTIS  2012   PAGE  40  CONFIDENTIAL  MIRANTIS  

Step 8b: Provision Scheduled

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Scheduler  publishes  message  to  the  compute  queue  (based  on  host  ID)  to  trigger  VM  provisioning  

©  MIRANTIS  2012   PAGE  41  CONFIDENTIAL  MIRANTIS  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Step 9a: Start VM Provisioning Nova  Compute  gets  message  from  MQ  

©  MIRANTIS  2012   PAGE  42  CONFIDENTIAL  MIRANTIS  

Nova Compute Drivers

Nova  Compute  

XCP  

VM  

VM  

VMWare  

PowerVM  

VM  

VM  

libvirt  

KVM  

VM  

VM  

Xen  

VM  

VM  

Qemu  

VM  

VM  

LXC  

Container  

Container  

Allows  mul>ple  hypervisor  types  per  cloud.    Libvirt  /  KVM  is  most  commonly  used  in  deployment  

Maintained  by  Microso_  

Maintained  by  IBM  

Bare  Metal  

Docker  

Container  

Container  

XenAPI  

Nai>ve  support  comes  in  Icehouse  PXE  Tilera  

Somewhat    experimental  

Maintained  by  Citrix  

HyperV  

VM  

VM  

ESXi  

VM  

VM  

vSphere  

VM  

VM  

©  MIRANTIS  2012   PAGE  43  CONFIDENTIAL  MIRANTIS  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Step 9b: Start VM Provisioning Nova  Compute  makes  rpc.call  to  Nova  Conductor  for  informa>on  on  VM  from  DB  

©  MIRANTIS  2012   PAGE  44  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Nova Conductor

•  Eliminates remote DB access (security)

•  Horizontal scalability: spawn multiple worker threads operating in parallel (performance)

•  Hides DB implementation/schema from the Nova Compute (upgrades)

•  Possible offloading of long-running operations from other services, not just Nova Compute

•  Beneficial for operations that cross multiple compute nodes (migration, resizes)

controller  node  

DB  

nova-­‐conductor  

compute  node  

nova-­‐compute  rpc.call()  

©  MIRANTIS  2012   PAGE  45  CONFIDENTIAL  MIRANTIS  

Step 10: Configure Network

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute  makes  a  call    to  Neutron  API  to  provision  network  for  the  instance  

©  MIRANTIS  2012   PAGE  46  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Neutron

• Provides a flexible API (POST / GET) for service providers or their tenants to manage OpenStack network topologies. •  Create networks, associate VMs, set routers, etc.

• Presents a logical API and a corresponding plug-in architecture that separates the description of network connectivity from its implementation.

• One can still choose to stay with nova-network (Essex approach) or to go with Neutron.

©  MIRANTIS  2012   PAGE  47  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Neutron Server

Neutron CLI Nova Horizon

Neutron  

Neutron Architecture

Queue

Neutron DB

Optional depending on plugin.

Neutron L3 Agent

HTTP  AMQP  SQL  3rd-­‐party  

Neutron  DHCP  Agent  

Local vSwitches

Neutron L2 Agent

Runs on each Compute Node. Optional depending on plugin.

L3&DHCP Agents Scheduler

Optional

Other  Network  Services  

FWaaS, VPNaaS, LBaaS, etc.

Neutron  Metadata  Agent  

Neutron Plugin

SDN Controller, etc.

Optional depending on plugin.

©  MIRANTIS  2012   PAGE  48  CONFIDENTIAL  MIRANTIS  

Step 10: Configure Network (Continued) Neutron  configures  IP,  gateway,  DNS  name,  L2  connec>vity,  etc.  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

©  MIRANTIS  2012   PAGE  49  CONFIDENTIAL  MIRANTIS  

Step 11: Request Volume

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

It  is  assumed  a  volume  is  already  created.  Nova  Compute  contacts  Cinder  to  get  volume  data.  Can  also  aWach  volumes  a_er  VM  is  built.  

©  MIRANTIS  2012   PAGE  50  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Open Stack Storage Concepts

•  Ephemeral storage: •  Persists until VM is terminated •  Accessible from within VM as local file system •  Used to run operating system and/or scratch space •  Managed by Nova

•  Block storage: •  Persists until specifically deleted by user •  Accessible from within VM as a block device (e.g. /dev/vdc) •  Used to add additional persistent storage to VM and/or run operating system •  Managed by Cinder

•  Object storage: •  Persists until specifically deleted by user •  Accessible from anywhere •  Used to add store files, including VM images •  Managed by Swift

©  MIRANTIS  2012   PAGE  51  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Cinder Resources

• Volumes: •  Persistent R/W Block Storage devices •  Can be attached to VMs as secondary storage •  Can be root store to boot VMs •  Can be attached only to one instance at a time •  Keep their state independent of instances

• Snapshots: •  Read-only point in time copy of a volume •  Can then be used to create a new instance

• Backups: •  An archived copy of a volume

©  MIRANTIS  2012   PAGE  52  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Cinder  

Cinder Architecture

Cinder DB

Queue

Cinder Volume

Backend Storage Devices Scheduler

Cinder API

Cinder CLI Nova Horizon HTTP  AMQP  SQL  3rd-­‐party  

Cinder Backup

Object Storage

©  MIRANTIS  2012   PAGE  53  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Cinder Volume Driver •  iSCSI:

•  Dell EqualLogic

•  EMC VMAX/VNX

•  Hitach HDS

•  HP 3PAR (StoreServ)

•  HP / Lefthand SAN (StoreVirtual)

•  Huawei T/Dorado/HVS

•  IBM Storwize family/SVC/XIV

•  LVM (Reference Implementation)

•  Nexenta

•  NetApp

•  SolidFire

•  VMware VMDK

•  Windows Server 2012

•  Zadara

•  GlusterFS NFS (volumes as sparse files)

•  IBM General Parallel File System (GPFS) (volumes as sparse files):

•  GPFS NSD

•  ATA over Ethernet (AoE):

•  Coraid

•  Fibre Channel:

•  NetApp

•  HP 3PAR (StoreServ)

•  Huawei T/Dorad/HVS

•  IBM Storwize family/SVC/XIV

•  VMware VMDK

•  NFS (volumes as sparse files):

•  NFS

•  Nexenta

•  NetApp

•  VMware VMDK

•  Zadara

•  XenAPI Storage Manager

•  RADOS Block Devices (RBD):

•  Ceph

•  Shared SAS:

•  VMware VMDK

•  Scale Out File System (SOFS) (volumes as sparse files):

•  Scality

•  VirtIO (Local raw storage) (volumes as sparse files)

©  MIRANTIS  2012   PAGE  54  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Cinder Backup Drivers

• Swift

• Ceph

•  IBM Tivoli Storage Manager (TSM)

©  MIRANTIS  2012   PAGE  55  CONFIDENTIAL  MIRANTIS  

Step 11: Request volume (Continued)

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute  sets  up  the  host  mount  if  needed  &  instructs  the  Hypervisor  to  use  vol.  as  a  new  block  device  

©  MIRANTIS  2012   PAGE  56  CONFIDENTIAL  MIRANTIS  

Step 12: Request VM Image from Glance

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute    requests  VM  image  from  Glance  via  Image  ID  

©  MIRANTIS  2012   PAGE  57  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Glance

"The Glance project provides services

for discovering, registering, and retrieving virtual machine images."

©  MIRANTIS  2012   PAGE  58  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Glance Summary

•  Images-as-a-Service.

• Can use multiple back-ends for image storage.

• Can store the same image in multiple locations.

• Supports multiple image formats.

©  MIRANTIS  2012   PAGE  59  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Glance API

Glance CLI Nova Horizon

Glance  

Glance Architecture

HTTP  AMQP  SQL  3rd-­‐party  

Glance Registry

Glance DB

Store Adapter

Swift

Cinder

File System

Amazon S3

HTTP

GridFS

Sheepdog

Ceph (RBD)

©  MIRANTIS  2012   PAGE  60  CONFIDENTIAL  MIRANTIS  

Step 13: Get Image URI from Glance

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

If  image  with  given  image  ID  can  be  found  -­‐  return    URI  –  HTTP  Get  URI  

©  MIRANTIS  2012   PAGE  61  CONFIDENTIAL  MIRANTIS  

Step 14: Direct Image File Copy

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute  can  download  image  using  URI,  given  by  Glance,  directly  from  Swift  

©  MIRANTIS  2012   PAGE  62  CONFIDENTIAL  MIRANTIS  

Step 14 alternative: Image Copy through Glance

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

To  leverage  Glance  Server  caching  mechanism  and  addi>onal  access  restric>on,  the  Image  copy  can  go  through  Glance  

©  MIRANTIS  2012   PAGE  63  CONFIDENTIAL  MIRANTIS  

Step 15: Start VM Rendering via Hypervisor

In  case  of  KVM  /  libvirtd  this  is  a  single  XML  VM  config  file  

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute  creates  a  command  to  Hypervisor  and  delegates  VM  rendering  to  Hypervisor.  

©  MIRANTIS  2012   PAGE  64  CONFIDENTIAL  MIRANTIS  

Step 16: VM is UP

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Nova  Compute  sends  a  message  to  Nova  Conductor  to  update  DB  with  VM  state  

©  MIRANTIS  2012   PAGE  65  CONFIDENTIAL  MIRANTIS  

Step 17: User is Happy

Nova  

Nova  DB  

Queue  Nova  API  

Scheduler  

Conductor  

UI:  Horizon  or  CLI  

SwiG    

Object  Store  

Proxy  Server  

Keystone      KeystoneAPI  

Keystone  DB  

Glance  Glance  API  

Glance  Registry  

Glance  DB  Neutron  

Neutron  DB  

Queue  Neutron  API  

Scheduler  

Plugin/Agent  

Compute  Node  Compute  Node  

Network  

VM  

Hypervisor  

nova-­‐compute  

Network  Node  DHCP/IPAM  

Router/GW  

Block  Storage  Node  Storage  

Cinder  

Cinder  API  

Scheduler  

Cinder  Backup  Cinder  DB  

Queue  

Cinder  Vol  

Ceilometer  Ceilometer  

API  Collector  

Agent  

Horizon  polls  Nova  API  for  VM  status  and  power  state,  which  is  taken  from  Database.  

©  MIRANTIS  2012   PAGE  66  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Recap:

•  Users log into Horizon and initiates VM creation •  Keystone authorizes •  Nova initiates provisioning and saves state to DB •  Nova Scheduler finds appropriate host •  Neutron configures networking •  Cinder provides block device •  Image URI is looked up through Glance •  Image is retrieved via Swift •  VM is rendered by Hypervisor

©  MIRANTIS  2012   PAGE  67  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

In the end Developers Win.

©  MIRANTIS  2012   PAGE  68  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Special Offer for OpenStack DC Meetup

Sign up for any OpenStack class by Mirantis

in Washington, DC in 2014 and save 10% off the ticket price.

To redeem your discount, use the code “DC_Meetup_2014”

Course schedule in Washington, DC:

OpenStack Bootcamp with Exam (OS110) Sep 30 – Oct 3

OpenStack Bootcamp with Exam (OS110) Nov 11 - 14

OpenStack Fundamentals (OS50) Nov 17

OpenStack Bootcamp II (OS200) Nov 18 - 20

For complete schedule, course description, and registration visit training.mirantis.com    

©  MIRANTIS  2012   PAGE  69  CONFIDENTIAL  MIRANTIS   ©  MIRANTIS  2013  

Questions and Comments?

Paul Roberts Twitter: pauljrob proberts@mirantis.com