State of the Stack v2

102
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution* * All unlicensed or borrowed works retain their original licenses State of the Stack v2 An Update on Why OpenStack is The New Linux. OpenStack Summit, Hong Kong November 7th, 2013 @randybias

description

This 2nd major State of the Stack address is a complete refresh of the spring 2013 edition, broadcast live on BrightTALK from the OpenStack Summit in Hong Kong. (Replay: https://www.brighttalk.com/webcast/10353/92159) Randy Bias, CEO and Co-founder of Cloudscaling examines the progress from Grizzly to Havana and delves into new areas like refstack, tripleO, bare metal server provisioning, the move from "projects" to "programs", and public/hybrid cloud compatibility. Check out the updated statistics on project momentum and look more closely at big upgrades in Havana, including OpenStack Orchestrate (Heat), which has the opportunity to change the game for OpenStack in the greater private and hybrid cloud game. We also discuss the "what is 'core'" debate and examine the idea that OpenStack is a kernel, not a complete cloud OS.

Transcript of State of the Stack v2

Page 1: State of the Stack v2

CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses

State of the Stack v2An Update on Why OpenStack is The New Linux.

OpenStack Summit, Hong KongNovember 7th, 2013

@randybias

Page 2: State of the Stack v2

Introduction

Page 3: State of the Stack v2

OpenStack Foundation Board of Directors

Prod. OpenStack pioneer, Cloudscaling:Part of OpenStack community since July 2010 (launch)Wins: KT, Internap, LivingSocial, Seagate & others

Advisor to dotCloud (docker.io)

Top 10 Cloud Computing Pioneer

Who

3

Page 4: State of the Stack v2

I run an OpenStack product company... opinions expressed here are mine

Not the foundation, not the community, just one POV

I believe the pioneers to emulate are:

I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switches

My Bias

4

Page 5: State of the Stack v2

5

1 What is OpenStack?

3 History & Momentum4 OpenStack Governance5 Stackology - OpenStack at 10km6 Stacking it Up - a dive into the projects

10 Summary

2 Why the Success?

8 Who’s using it and how?7 Stack Gaps

9 RightScale - State of the Cloud Report

Page 6: State of the Stack v2

What is OpenStack?

Page 7: State of the Stack v2

OpenStack From 10km

7

Networking

OPENSTACK CLOUD OPERATING SYSTEM

Standard Hardware

Compute Storage

Your Applications

OpenStack Dashboard

OpenStack Shared Services

APIs

Page 8: State of the Stack v2

OpenStack Mission

8

"To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to

implement and massively scalable."

Code Community

Page 9: State of the Stack v2

OpenStack Foundation Mission

9

The OpenStack Foundation is an independent body providing shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.

The ubiquitous cloud computing platform

Page 10: State of the Stack v2

The Battle is Over (open src)

10

OpenStack CloudStack Eucalyptus OpenNebula

Source: trends.google.com

OpenStack Launch

Page 11: State of the Stack v2

Battle is Nearly Over (closed src)

11

OpenStack vSphere vCloud

Source: trends.google.com

Page 12: State of the Stack v2

Linux 2000 vs. Linux 2009

12

Is this OpenStack’s Trajectory?

Operating system family market share

2000 2009

Unix Linux

Linux Unix

Source: Linux Magazinehttp://www.linux-mag.com/id/7749/

Linux

UNIX

BSD

Windows/Other

Mixed

Page 13: State of the Stack v2

Fastest Growing Global Open Source Community

13

COMPANIES

TOTAL DEVELOPERS AVERAGE MONTHLY CONTRIBUTORS

TOTAL CODE CONTRIBUTIONS

1,036 >400 70,137

275TOP 10 COUNTRIES

12,439INDIVIDUAL MEMBERS

“OpenStack appears to be a more advanced or more modern open source project than some of its predecessors because it's a highly coordinated effort.”

– Charlie BabcockInformation Week

COUNTRIES

72+United States, China, India, Great Britain, Australia, France, Russia, Canada, Ireland, Germany

Page 14: State of the Stack v2

14

Havana StatsCONTRIBUTORS

COMMITS / DEV INCREASE IN NEW FEATURES

TEST CLOUDS DEPLOYED DAILY

~22 70% >700

910 (+70%)TOP 10 CONTRIBUTING COMPANIES

+20,000COMMITS MERGED

“OpenStack appears to be a more advanced or more modern open source project than some of its predecessors because it's a highly coordinated effort.”

– Charlie BabcockInformation Week

NEW FEATURES

392Canonical, Dreamhost, eNovance, HP, IBM, Intel, Mirantis, OpenStack Foundation, Rackspace, RedHat, Suse, and Yahoo!

Page 15: State of the Stack v2

Why the Success?

Page 16: State of the Stack v2

Who or What Should We Thank?

OpenStack Foundation & CommunityEsp., all of the companies who realized this could be big

The Big Enterprises for Driving Interest

Maturity of Community SDLCNo other similar project has this SDLC+velocity

16

1

2

3

Page 17: State of the Stack v2

OpenStack Development Maturation

17

Regular

Releas

e

Cycle

Professional Open

Source Leaders

Advance

d CI &

Testi

ngCentralized Shared

Libraries (Oslo)

SDLC

Page 18: State of the Stack v2

Regular Release CycleEvery 6 months, we coordinate & integrate:Thousands of patches & commitsAcross hundreds of developersWhile also increasing # of projectsIn last 3 yrs: grew from 2 to 9 “integrated” or “core” projects+4 more new projects in “incubation” as of Havana

No other related project is capable of thisNo other related project has this many capabilities or is this complex

18

Page 19: State of the Stack v2

Professional Open Source Leaders

Experienced open source companies:RedHat, IBM, HP, Mirantis, etc.

Dev leaders w/ open source histories:Nova - Russell Bryant (Asterisk)Oslo - Mark McLoughlin (KVM, GNOME, Linux kernel, Java)TripleO & CI system - Monty Taylor (MySQL)... etc.

Very different from origins

19

Page 20: State of the Stack v2

Advanced Continuous IntegrationMassive Effort à Improved QualityParallel test pipelining w/ grouped, related changesets

Gated CommitsAll Code Has to Jump Through GatesTempest Test Framework

Service Level Testing, Regressions, Full Behavorial Tests, etc.

Code Reviews & Continuous IntegrationJenkins, GerritAt scale: jenkins.openstack.org

More than 700 end-to-end tests every day

20

Page 21: State of the Stack v2

Massive Test Suite

21

0

5000

10000

15000

20000

Nova Neutron CinderKeystone Glance OsloHeat Swift Tempest

0

175

350

525

700

7/2012 12/2012 3/2013 7/2013

Tempest Test Count

Total Tests (Unit + Tempest) Tempest Tests in One Year

Page 22: State of the Stack v2

Centralized Shared LibrariesProject OsloLess interest in shared code and reuse initiallyRun by experienced leader, Mark McLoughlinWhere all common code goes

Allows shared code and reuseEasier re-factoringCore database code recently centralizedoslo.db library

22

Page 23: State of the Stack v2

Take Away: OpenStack is Well Organized

Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q3http://www.qyjohn.net/?p=3373

23

“[the coordinated releases are] an indicator that the OpenStack project is well organized in terms of sub-project management.”

Page 24: State of the Stack v2

History & Momentum

Page 25: State of the Stack v2

A Short History of OpenStack

Launch!

25

Growing Pains & Early

Adopters

Rise of the Foundation

Breakout

2010

2011

2012

2013

+

Page 26: State of the Stack v2

Incredible Industry Support

26

The top 3 vendors in every single category support OpenStack

Top 3 Router Vendors Top 3 Blade Vendors Top 3 Linux Vendors

Top 3 x86 Vendors

Top 3 HypervisorsTop 3 Storage VendorsTop 3 Switch Vendors

Page 27: State of the Stack v2

Developer Growth

27

Contributors per month (ohloh)

Page 28: State of the Stack v2

Developer Growth Comparison

Contributors per month (ohloh)

28

Page 29: State of the Stack v2

Dev Growth by Git Contributors

Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q3http://www.qyjohn.net/?p=3373

29

Page 30: State of the Stack v2

Accumulated Community

Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q3http://www.qyjohn.net/?p=3373

30

Page 31: State of the Stack v2

Growth by Domain (company - roughly)

Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q3http://www.qyjohn.net/?p=3373

31

For the CloudStack projects, influence from Citrix is quite obvious, over 48% of github.com commits come from accounts belonging to citrix.com and cloud.com.

Page 32: State of the Stack v2

Summit Growth

32

0

1,000

2,000

3,000

4,000

Austin San Antonio Santa Clara Boston San Francisco San Diego Portland Hong Kong*

*estimated

Page 33: State of the Stack v2

Job Growth vs. Others

33

Page 34: State of the Stack v2

OpenStack Project Goverance

Page 35: State of the Stack v2

Creating a Technical MeritocracyLaunch w/

ProjectOversight Committee

35

July 2010 Mar 2011 Oct 2011 Apr 2012

ProjectPolicy Board

Tech Comm Tech Comm

Foundation Announced

Board ofDirectors

Tech Comm

Rackspace Control

CommunityControl

Page 36: State of the Stack v2

Philosophical Approach

36

Ensure an inclusive process for contributors

https://wiki.openstack.org/wiki/Governance/Foundation/Structure

Companies Individuals

Users

Page 37: State of the Stack v2

8 8 8

Foundation Overview

37

Direct Governance Model

Foundation Board of Directors

Technical Committee

UserCommittee

Represents business interests

Represents developers

Represents users

https://wiki.openstack.org/wiki/Governance/Foundation/Structure

13 TBD

50 Corp Sponsors

200+ Supp. Orgs

8 PlatinumMembers

10,000+ Ind Members

16 Gold Members

Page 38: State of the Stack v2

Foundation Board of Directors

38

Represents the business interests of the OpenStack Project

https://wiki.openstack.org/wiki/Governance/Foundation

Foundation Board of DirectorsProvides strategic and financial oversight to promote OpenStack

8 8 8

8 PlatinumMembers

10,000+ Ind Members

16 Gold Members

50 Corp Sponsors

200+ Supp. Orgs

Page 39: State of the Stack v2

Technical Committee

39

Represents developers... technical people making technical decisions

https://wiki.openstack.org/wiki/Governance/Foundation/TechnicalCommittee

Members are elected by individual members every 6 months for a staggered 1 year term (6 Fall/7 Spring)

Public IRC meetings held weekly

Anyone can join but only the 13 TC Members can vote

TC is independent, meritocratic and able to change its structure and processes

Technical Committee

Program1 P2 ...PTL PTL

Active ProgramContributors

13 Members

10,000+ Ind Members

Page 40: State of the Stack v2

OpenStack User Committee

40

Represents OpenStack users

https://wiki.openstack.org/wiki/Governance/Foundation/UserCommittee

The structure of the User Committee is still being defined

Mission is to:• Consolidate user requirements

for the board and TC• Provide guidance for the

development teams• Track OpenStack deployments

and usage• Work with user groups worldwide

Page 41: State of the Stack v2

OpenStack Programs

41

Efforts Essential to the Completion of the OpenStack Mission

https://wiki.openstack.org/wiki/Programs

Elected Project Technical Leads (PTLs) organize the work of teams and make project-level decisions for each program

PTLs are elected by Active Program Contributors (APCs) every 6 months

Programs are under the oversight of the Technical Committee

NovaSwift

GlanceCinder

KeystoneHorizonNeutron

CeilometerHeat

...

Page 42: State of the Stack v2

Stackology

Page 43: State of the Stack v2

What is OpenStack?

43

1 A open source component framework for IaaS clouds

3 Analogous to the Linux kernel (very tunable)

2 Delivering compute, networking & storage services

Page 44: State of the Stack v2

CLI toolsDashboard Other tools

ComputeNetworkingOrchestration

Hypervisor(s) QueuingDatabase

/ KVS/ Cache

External Block

Provider

Physical Network Provider

Provisioning

Log Aggregation

Health Monitoring

etc.

REST Meter Data

REST

SQL, etc. Varies Varies Varies AMQP/0MQ

Topology & Metadata

MeteringREST

REST REST

DNS

ImageManagement

IdentityManagement

REST

Block Storage

Object Storage

OpenStack (m)Architecture Slide

44

UI Layer

ElasticServices

Layer

Oth

er s

tuff,

you

pro

babl

y ne

ed/w

ant

Data &Resource

Layer

SharedServices

Layer

Page 45: State of the Stack v2

CLI toolsDashboard Other tools

ComputeNetworkingOrchestration

Hypervisor(s) QueuingDatabase

/ KVS/ Cache

External Block

Provider

Physical Network Provider

Provisioning

Log Aggregation

Health Monitoring

etc.

REST Meter Data

REST

SQL, etc. Varies Varies Varies AMQP/0MQ

Topology & Metadata

MeteringREST

REST REST

DNS

ImageManagement

IdentityManagement

REST

Block Storage

Object Storage

OpenStack (m)Architecture Slide

44

UI Layer

ElasticServices

Layer

Oth

er s

tuff,

you

pro

babl

y ne

ed/w

ant

Data &Resource

Layer

SharedServices

Layer

Page 46: State of the Stack v2

CLI toolsDashboard Other tools

ComputeNetworkingOrchestration

Hypervisor(s) QueuingDatabase

/ KVS/ Cache

External Block

Provider

Physical Network Provider

Provisioning

Log Aggregation

Health Monitoring

etc.

REST Meter Data

REST

SQL, etc. Varies Varies Varies AMQP/0MQ

Topology & Metadata

MeteringREST

REST REST

DNS

ImageManagement

IdentityManagement

REST

Block Storage

Object Storage

OpenStack (m)Architecture Slide

44

UI Layer

ElasticServices

Layer

Oth

er s

tuff,

you

pro

babl

y ne

ed/w

ant

Data &Resource

Layer

SharedServices

Layer

Page 47: State of the Stack v2

Projects Description Layer AWSEquivalent Codenames

Dashboard Self-service, role-based web interface for users and administrators UI Console Horizon

Compute Provision and manage large pools of on-demand computing resources

Elastic Service EC2 Nova

Block Storage Volumes on commodity storage gear, and drivers for turn-key block storage solutions

Elastic Service EBS Cinder

Object Storage

Petabytes of reliable storage on standard gear

Elastic Service S3 Swift

Networking L2-focused on-demand networking with some L3 capabilities

Elastic Service VPC Neutron

Orchestration Application orchestration layer that runs on top of and manages OpenStack Compute

Elastic Service

CloudFormation, CloudWatch Heat

Metering Centralized metering data for all services for integration to external billing

Shared Service N/A Ceilometer

Identity Multi-tenant authentication system that ties to existing stores (e.g. LDAP) and Image Service

Shared Service None Keystone

Image Management

Upload, download, and manage VM images for the compute service

Shared Service

VM Import/Export Glance

45

Page 48: State of the Stack v2

Incubated à Integrated à Core

46

https://wiki.openstack.org/wiki/Governance/NewProjects

Ensures projects meet a high standard of usefulness and quality

Page 49: State of the Stack v2

OpenStack is Expanding (Grizzly)

47

Direction of Project/Program Induction

Nova

Swift

Glance

Keystone

Horizon

Neutron

Cinder

Ceilometer

Heat

Incubation Integration Core*

Greater O

penStack Ecosystem

Page 50: State of the Stack v2

OpenStack is Expanding (Havana)

48

Direction of Project/Program Induction

Nova

Swift

Glance

Keystone

Horizon

Neutron

Cinder

Ceilometer

Heat

Incubation Integration Core*

TripleO

Ironic

Marconi

Trove

Greater O

penStack Ecosystem

Page 51: State of the Stack v2

OpenStack is Expanding (Icehouse)

49

Direction of Project/Program Induction

Nova

Swift

Glance

Keystone

Horizon

Neutron

Cinder

Ceilometer

Heat

Incubation Integration Core*

TripleO

Ironic

Marconi

Trove

Savanna

Greater O

penStack Ecosystem

Page 52: State of the Stack v2

Stacking It Up

Page 53: State of the Stack v2

Brief Word On the Architecture

51

Bi-furcated set of servicesSome pluggable, some monolithicSome distributed, some much less soSome inherently HA, some not

Pluggable, distributed services use RPCInternal RPC mechanism is (a)sync message passingAllows remote method invocation (as expected)

Page 54: State of the Stack v2

Keystone

Bi-furcated Approach

52

Nova Swift

GlanceCinder

Pluggable Monolithic

Ceilometer

Heat

Horizon

Neutron

Savanna

Trove

Marconi

Ironic

TripleO

?

Page 55: State of the Stack v2

OpenStack RPC

53

{ 'oslo.version': '2', 'oslo.message': json ( { 'method': 'method_name', 'args': { 'keyword': 'value' } } )}

nova-api nova-scheduler

Remote Procedure Call(invoked via (a)synchronous message passing)

Page 56: State of the Stack v2

A Quick Note of Thanks

54

These diagrams would not have been possible without the prior work of:Ken Pepple, Solinea (@ken_pepple)Dina Belova, Mirantis

... and the help of several Cloudscalers:Eric Windisch (@ewindisch)Joe Gordon (http://github.com/jogo)Dan Sneddon (@dxs)Joseph Glanville (@jpgvm)

Page 57: State of the Stack v2

Caveat Emptor

55

The focus for these diagrams was ease of reading, not accuracy.

See Ken Pepple’s originals or the code if you need truth.

That being said, our team tried really hard for accuracy.

Blame me for any errors.

Page 58: State of the Stack v2

Architecture Diagrams Legend

56

Page 59: State of the Stack v2

CLI toolsDashboard Other tools

ComputeNetworkingOrchestration

Hypervisor(s) QueuingDatabase

/ KVS/ Cache

External Block

Provider

Physical Network Provider

Provisioning

Log Aggregation

Health Monitoring

etc.

REST Meter Data

REST

SQL, etc. Varies Varies Varies AMQP/0MQ

Topology & Metadata

MeteringREST

REST REST

DNS

ImageManagement

IdentityManagement

REST

Block Storage

Object Storage

OpenStack (m)Architecture Slide

57

UI Layer

ElasticServices

Layer

Oth

er s

tuff,

you

pro

babl

y ne

ed/w

ant

Data &Resource

Layer

SharedServices

Layer

Page 60: State of the Stack v2

OpenStack Compute (Nova)

58

Page 61: State of the Stack v2

Compute Thoughts

59

Nova still runs best w/ KVMDo we need another hypervisor? What’s the biz case?ESX support is a mess

Multiple Availability Zones still not solvedCells are for making one AZ bigger

complex, tight-couplingWe need a clean sharding mechanism for AZes

also what about Cinder and Neutron?

Cinder/Neutron integr. needs examiningMore information needs to be able to be passed back

Page 62: State of the Stack v2

Compute (Networking) Thoughts

60

nova-network still requiredNeutron has been L2 focused & L3 gap still exists

centralized “single-host” networking is #faildecentralized “multi-host” is more #failnova-conductor security for hypervisor obviatedmetadata-api, nova-network, or Neutron on all HVs?

security implications

reconciling Neutron and nova-network?Neutron needs more L3 capability, but ...

do we always want Neutron in every deployment?

Page 63: State of the Stack v2

OpenStack Block Storage (Cinder)

61

Page 64: State of the Stack v2

Block Storage Thoughts

62

Default “nova-volume” func. is too minimalWhen people think Block Storage service they assume:

Persistent, Network-based, & Performant - it isn’t

Cinder scheduler needs info from NovaAssuming you want to do anything interesting

Point of lock-in since default isn’t usefulYou have to place a bet on a block storage solutionThese are expensive, experimental or proprietaryRely on vendor not community to help w/ problems

Page 65: State of the Stack v2

OpenStack Networking (Neutron)

63

Page 66: State of the Stack v2

Networking Thoughts

64

Default networking functionality is minimalThe APIs have been L2-centricL3 functionality is same as existed with nova-network

Same architecture, same basic layout, with all of the downsides

Needs a Neutron plugin for full func.Can’t run more than one plugin at a time per functionNot clear how many production deployments there are:

http://lists.openstack.org/pipermail/openstack-operators/2013-September/003584.html

Good news is that this area is hotSo hopefully this is unstuck soon

Page 67: State of the Stack v2

OpenStack Object Storage (Swift)

65

Page 68: State of the Stack v2

Object Storage Thoughts

66

Strength: massive concurrency, proven scaleSwift needs additional attentionToo much focus on Nova and Neutron as the “new shiny”Container replication is a bad hackOriginally most mature of the projects; but has lagged

Good news: area has many new playersSeagate, SwiftStack, RedHat, IBM, eNovance, Mirantis, and HP all deeply involved

Major new features coming:Global clusters for multi-site replicationFeatures for prioritized replicationPluggability for support of DFS like GlusterFS (RedHat)

Page 69: State of the Stack v2

OpenStack Image Mgmt (Glance)

67

Page 70: State of the Stack v2

Image Mgmt Thoughts

68

Still not clear why this is standalone projectReally a sub-function of Compute

Semi-pluggable (but not really)Uses different backends for image storage

To be really useful it needs more features:P2V, V2V, and other image conversionAbility to slipstream PV drivers into imagesConvert from popular formats: OVF, AMI, etc.

Page 71: State of the Stack v2

OpenStack Identity (Keystone)

69

Page 72: State of the Stack v2

Identity Mgmt Thoughts

70

Mixed identity / schizophreniaVerifies identity, authorization, AND service registry

Service registry is one of manyNova, Cinder, et al have their own internal registries

Slows everything downSee LivingSocial presentation from Folsom SummitMitigated with memcache assuming you turn it on

OpenStack needed to reinvent wheel here?Could have just been LDAP with a schema + caching

Page 73: State of the Stack v2

OpenStack Dashboard (Horizon)

71

Updates

Model

View Controller

User

Manipulates

Sees Uses

Page 74: State of the Stack v2

Dashboard Thoughts

72

It’s gotten waaay better!Same UI for end-user and admin is bad idea

CloudStack did this and it was a messThe workflows and views are too differentSecurity considerations exist

General lag: many things aren’t accessible e.g. Heat, TripleO, Marconi, Trove, etc.Can be a 6 month cycle time for new feature UI support

Need better docs on extending, w/o harmCustomers, product companies, SPs all want to modifyAllow for customizations, while supporting upgrades, etc.

Page 75: State of the Stack v2

OpenStack Metering (Ceilometer)

73

Page 76: State of the Stack v2

Metering Thoughts

74

Metering systems are hardBad or incomplete data for SPs is existentialMetering system should be *very* baked (is 1yr enough?)

No tokenized meter dataInstance hours not enoughHow do you bill for Windows, Oracle, RHEL licenses?Tokens stack: size of instance, OS and app licenses, etc.

Needs to get flow data from edge switchesNetflow and/or Sflow support for physical switches

Data from the vSwitches is not the best source in the real world

Page 77: State of the Stack v2

OpenStack Orchestration (Heat)*

75

* Source: http://www.slideshare.net/dbelova/openstack-heat-slides

Page 78: State of the Stack v2

Orchestration Thoughts

76

Huge potentialAdds additional AWS func: CloudWatch, CloudFormationProvides clean templates for stacks, which means:

OpenStack on OpenStack (OoO) for testing, etc.

First primary project that rides “on top”Clear differentiator over other projects

Initiative: Heat templates for Ref ArchVendors, customers, etc. could feed to prov systems:Crowbar, Piston, Cloudscaling, etc.

Page 79: State of the Stack v2

Strengths:De facto winnerIncredible communityMature SDLCMini-benevolent dictators in PTLsSuper flexible framework

Can build any kind of cloud

SWOT: OpenStack

77

Opportunities:Build an SQL92 base for cloud compute (see Threats)

refstackPublic cloud compatibility as first order initiative

AWS, GCE, and vCloudHybrid-first strategy

Weaknesses:No single benevolent dictatorInteroperability will be difficult

Not impossible, *difficult*Flexibility limits interoperability

Threats:Splintering, fragmentation, and customizationCustomer DIY failures

Flexibility allows you to shoot yourself in the foot easily

Forking or ivory tower thinking

Page 80: State of the Stack v2

Stack Gaps

Page 81: State of the Stack v2

What’s In a Complete Cloud OS?

79

OpenStack Relationship Who? OpenStack

ScoreEcosystem

ScoreVendor Target*

User Interface(s) Horizon, CLI, ... OpenStack, Vendor, Ecosystem 4 2 4

Elastic Resource Management

Nova, Swift, Neutron, Cinder, ...

OpenStack, Vendor, Ecosystem 4 1 4

Service Discovery Scattered:Nova, Keystone, ... OpenStack, Vendor 2 0 4

AAA ControlsKeystone (authen/author), various projects (ACLs)

OpenStack 2 0 4

HW/SW Life Cycle Management N/A Vendor, Ecosystem 1 2 4

Service Management N/A Vendor 0 1 4Health & Logging N/A Vendor, Ecosystem 0 0 4Topology & Inventory N/A Vendor 0 0 4Hardware Certifications N/A OpenStack,

Vendor, Ecosystem 1 1 2*** We’re all _trying_ to close this gap ** It’s a hard problem no one will solve individually

Page 82: State of the Stack v2

Your Basic Choices

Download OpenStack and DIY

OpenStack Distributions

Turn-key Products or Systems powered by OpenStack

80

1

2

3

Page 83: State of the Stack v2

OpenStack Deployment Options

81

Deploy a Turnkey System

(product)

Hire a 3rd Party

(distro, src)

Do It Yourself

(distro, src)

Do Nothing

Time to Production 4 2 1 0

Cost of Ownership 4 2 1 0

Minimize Resources Needed 4 0 1 4

Support 4 1 0 0

Upgradability & Enhancements 4 2 2 0

Customization / Complexity 2 3 1 0

Page 84: State of the Stack v2

The Downside of DIY

Completing Cloud OS is on youIt’s not like downloading MySQLBuilding your own custom Linux? Really?

Impediment to innovation1) You can’t upgrade to latest because you customized2) You can’t customize because you need to upgrade

Silo & empire building again? (NOT cloud)Training? Support? Hardware variation?

82

Trunk!8%!

Grizzly!15%!

Folsom!47%!

Essex!25%!

Diablo!5%!

Version!

Page 85: State of the Stack v2

Who’s Using It?

Page 86: State of the Stack v2

OpenStack 2nd User Survey

383#

182#

27#25#

191#

Top$10$countries$

13$14$23$24$24$25$39$57$63$

336$

IT$BR$DE$AU$GB$CA$FR$CN$IN$US$

387  Deployments

822 survey  responses

539  companies

216  UG  members

Page 87: State of the Stack v2

Organizations & Drivers

85

Finance'1%'Healthcare'1%'Retail'1%'Manufacturing/Industrial'2%'

Government'/'Defense'2%'

Film/Media'3%'

Unspecified'3%'

Other'6%'

TelecommunicaEons'7%'

Academic'/'Research'11%'

InformaEon'Technology'63%' Other'

19%'

Industries*

4'48'

52'65'

86'87'

95'95'95'98'

Other'AMracEng'talent'

Control'Time'to'market'

Ability'to'innovate,'compete'Flexibility'of'underlying'technology'choices'

Avoiding'vendor'lockSin'Open'technology'

OperaEonal'efficiency'Cost'savings'

Business*Drivers*

1S20'employees'26%'

21S100'employees'16%'

101'to'500'employees'

14%'

501'to'1,000'employees'

5%'

1,001'to'5,000'employees'

10%'

5,001'to'10,000'employees'

8%'

More'than'10,000'employees'

21%'

Organiza2on*size*

Page 88: State of the Stack v2

OpenStack Deployments

86

Hosted''Private'Cloud'

17%'

On5Premise'Private'Cloud'

58%'

Public'Cloud'15%'

Hybrid'Cloud'6%'

Community'Cloud'1%'

unspecified'3%'

AusAn

'

Bexar'

Cactus'

Diablo'

Essex'

Folsom

'

Grizzly'

Havana'

Trun

k'

April'2013'

Nov'2013'

Openstack'Compute'

Openstack'Image'Service'

Openstack'IdenAty'Service'

Openstack'Dashboard'

Openstack'Network'

Openstack'Block'Storage'

Openstack'Object'Storage'

Ceilometer' Heat' OpenStack'Bare'Metal''

OpenStack'DbaaS'

April'2012'

Nov'2013'

Page 89: State of the Stack v2

OpenStack Deployments

87

178$

173$

165$

Proof%of%Concept%

Dev/QA%

Produc3on%

Stage%

258$181$

168$173$

122$86$

3$

Dashboard$Snapsho4ng$to$new$images$

Live$migra=on$Object$storage$

EC2$compa=bility$API$S3$compa=bility$API$

OCCI$compa=bility$API$

Features%

LVM%

CEPH%

Puppet%%Chef%

KVM%

XEN$

OpenVswitch%

Linux%Bridge%

Ubuntu%

Centos%

+40+56

AWS API usage as % is holding steady

Page 90: State of the Stack v2

OpenStack Deployments

88

1"1"1"2"2"3"3"3"3"3"3"4"4"5"5"5"7"8"8"9"12"13"15"32"37"

71"

Mellanox"Other"iSCSI"

EqualLogic"XenAPI"Storage"

Custom"Huawei"Scality"

Sheepdog"ZFS"

Zadara"HP"3PAR"

HP"LeLHand"Coraid"

Storwize"XIV"

SolidFire"SAN/Solaris"

Xenapi"Nexenta"SAN/HP"

Windows"EMC"

GlusterFS"NetApp"

NFS"

260"

77"

138"

Other"

Ceph"

LVM"

Storage(Driver(

86"

35"

72"

Other"

Chef"

Puppet"

Deployment(tool(

1"1"1"1"1"2"2"2"2"2"8"8"

16"39"

Anvil"CFEngine"

FAI"Foreman"

None"Ansible"

Fuel"Other"

StackOps"Substratum"

Crowbar"SaltStack"PackStack"DevStack"

145"

50"

312"

Other"

xen"

kvm"

Hypervisor(

1"1"4"5"

13"13"21"23"23"

41"

Docker"PowerVM"

Other"OpenVZ"

Bare"Metal"hyperv"

lxc"QEMU"

xenserver"esx"

49"

49"

111"

Other"

Centos"

Ubuntu"

OS(

1"

1"

2"

3"

3"

3"

6"

9"

21"

FreeBSD"

Other"

Scien]fic"Linux"

Fedora"

SUSE"Linux"

openSUSE"

Debian"

Windows"

RHEL"

125"

101"

155"

Other"

Linux"Bridge"

OpenVswitch"

Network(Driver(

2"2"2"2"3"3"3"4"4"6"7"12"15"

22"38"

Arista"Custom"

Mellanox"Modular"Juniper"

MidoNet"PLUMgrid"

NEC"Other"Ryu"

Big"Switch"Brocade"Hyper`V"Nicira"Cisco"

2"

13"

20"

103"

171"

Templated"

KVS"

PAM"

LDAP"

SQL"

Iden9ty(Driver(

Page 91: State of the Stack v2

OpenStack Deployments

89

Production Scale#"of"Cores"

63"

34"

11"17"

8" 10"

1"100$

101"500$

501"1,000$$

1,001"5,000$

5,001"10,000$

More$than$10,000$

64"

27"

15"22"

5"9"

1"100$

101"500$

501"1,000$

1,001"5,000$

5,000"10,000$

>$10,000$

#"of"Instances"

34"

14"

11"

4"

3"

1"

1"10,000$

10,001"100,000$

100,001$to$1$M$

1$M$to$100$M$

100$M$to$500$M$

More$than$500$M$

#"of"Stored"Objects"

46"

39"

11"

11"

0=10"TB"

11=100"TB"

100=500"TB"

>"500"TB"

Block"Storage"Size""

40" 43"

21"

9"

less"than"100"

101"to"1,000"

1,001"to"10,000"

More"than"10,000"

#"of"IP""

97"

14"

5"

17"

11"

1=50"

101=500"

501=1,000"

51=100"

More"than"1,000"

#"of"nodes"

Page 92: State of the Stack v2

OpenStack Deployments

90

Private vs Public Production24#

14#

3#

6#

2# 2#

7#

1# 1# 1# 1#

1(100# 101(500# 501(1,000# 1,001(5,000# 5,000(10,000# More#than#10,000#

unspecified#

Private#

Public#

#instances#

36#

7# 4# 1# 3#7#

18#

1# 2# 0# 3# 4#

1(50#

51(100#

101(500#

501(1,000#

>1,000#

unspecified#

#nodes#19#

11#

2# 3#

23#

9#

5# 4#1#

9#

0(10#TB# 11(100#TB# 100(500#TB# >500#TB# unspecified#

Size#Block#Storage#

Page 93: State of the Stack v2

Who Hacked on “core” for Havana?

Mirantis Stackalytics - All contributors for “core” in Havanahttp://www.stackalytics.com/?release=havana&metric=loc&project_type=core&module=&company=&user_id=

91

Page 94: State of the Stack v2

RightScaleState of the Cloud

Report

Page 95: State of the Stack v2

Enterprises Want Multi-Cloud/Hybrid

93

Source: RightScale State of the Cloud Report 2013

Single private 5%

Single public 11%

No plans 7%

Multi-public 15%

Multi-private 15%

Hybrid cloud 47%

Multi-cloud 77%

Enterprise Cloud Strategy 1000+ employees

Page 96: State of the Stack v2

Those Using Pub Want Hybrid More

94

Source: RightScale State of the Cloud Report 2013

Single private 2%

Single public 11%

No plans 1%

Multi-public 22%

Multi-private 13%

Hybrid cloud 53%

Multi-cloud 88%

Enterprise Cloud Strategy of Public Cloud Users 1000+ employees

Page 97: State of the Stack v2

AWS is Dominant Public Cloud

95

0% 20% 40% 60% 80% 100%

Windows Azure

GCE

RAX

AWS

% of Respondents

Public Cloud Usage

Running apps

Experimenting

Source: RightScale State of the Cloud Report 2013

Page 98: State of the Stack v2

OpenStack is Dominant Private Cloud

96

0% 20% 40% 60% 80% 100%

Eucalyptus

CloudStack

OpenStack

% of Respondents

Private Cloud Usage

Running apps

Experimenting

Source: RightScale State of the Cloud Report 2013

Page 99: State of the Stack v2

A Hybrid-First Cloud Strategy

97

1) Embrace public cloud architecturesAWS, GCE, RAX, Azure, and possibly vCloud

2) Expand work on refstack & TempestPublic cloud interoperability & hybrid cloud

3) Embrace & extend existing AWS testseutester, aws-compat, OpenStack tempest, NetflixOSS

Page 100: State of the Stack v2

Summary

Page 101: State of the Stack v2

OpenStack is Winning

99

1

2

3

4

5

New Cloud Kernel for the DatacenterLinux kernel to the other’s BSD kernel

Runaway Train... in Terms of Hype, Adoption, AND Development

Rapidly Expanding2 -> 14 projects in ~3 years

Hundreds of deployments> 165 in production (that we know of)

Get involvedhttp://is.gd/openstack

Page 102: State of the Stack v2

Q & A

http://simplicityscales.com/engineering blog

100

Randy BiasCEO & Co-founder, CloudscalingDirector, OpenStack Foundation@randybias