Post on 07-Aug-2015
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licenses
SOTS v4State of the Stack May 20th, 2015
OpenStack Summit, Spring 2015
@randybias
With significant help from many Cloudscalers and EMCers. Thank you!
The Randy Bias• Built big clouds; production clouds
• An OpenStack Original
• part of launch in 2010, on Foundation Board since formation
• built some of the largest and earliest OpenStack clouds
• Top <insert number here> cloud/twitter/pioneer/visionary
• you pick…
2
Fastest Growing Open Src Community
4
COMPANIES
TOTAL DEVELOPERS AVERAGE MONTHLY CONTRIBUTORS
TOTAL CODE CONTRIBUTIONS
3,654 >600 125,000+
502TOP 10 COUNTRIES
27,398INDIVIDUAL MEMBERS
COUNTRIES
140+United States, India, China, United Kingdom, France, Russia, Canada, Germany, Japan, Australia
Now That’s Something
5
OpenStackCloudStackEucalyptusVMware vSphere
Source: trends.google.com
Active Contributors in The Community
6
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1http://www.qyjohn.net/?p=3801
Monthly Commit Volume
7
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1http://www.qyjohn.net/?p=3801
Accumulated Developers
8
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1http://www.qyjohn.net/?p=3801
13
Infrastructure as a ServiceCompute
Nova Ironic
Magnum
NetworkNeutron (LBaaS)
(VPNaaS) (FWaaS)
StorageSwift
Cinder Manila
Cloud Management
Telemetry Ceilometer
Deployment Triple O
Orchestration Heat
Test Suites Tempest
Rally
Advanced Services (Consume IaaS)
Image Management: Glance
Data Processing
Sahara
Key Management
Barbican
DNS Management
Designate
Database Management
Trove
Message Queue Zaqar
Service Catalog Murano
Workflow Management
Mistral
Policy Management
Congress
Common/Shared: Identity: Keystone Common Libraries: Oslo
User/Admin
UI API CLIKilo
– Me / Ako / Moi / Yo *
“OpenStack is at risk of collapsing under its own weight.”
15* http://www.cloudscaling.com/blog/openstack/the-future-of-openstack-is-now-2015/
Lots of Improvements• Product WG formed
• Create an aggregation point for longer term planning, bring user feedback into process, prioritization of blueprints, lobbying TC and PTLs for work queues, “funding” of key blueprints, etc.
• Integrated release & 6-month cycle reformed to “Big Tent” approach
• No more forced 6-month integration, more project autonomy, encouragement of 3rd party integration testing of drivers, tagging for release, etc.
16
Product Working Group
17
User Committee N+3 members: 3 selected by the board, the TC and an additional nominated representative. An additional N
members elected by the user community.
Enterprise
Focused teams to gather user requirements from segments and represent them
Telco / OPNFV
Application Ecosystem
Large Deployments
API Working Group
Working Groups to address a particular requirement set. These WGs should have a target set of deliverables and conclude when those are met. Maintenance should be a
function of the regular workflows.
Logging
Ops Tools
Monitoring
HPC
Product Working Group Gather requirements from both sets of WGs (Segment and Requirement Oriented) above in the form of user
stories, work with cross-project team to populate blueprints from user stories across projects, work to identify developers to help complete blueprints, communicate with project PTLs and core team to collect feedback on future directions, and compile this data into a multi-release roadmap that is publicly available. In summary,
facilitate a feedback loop between projects, user community, and working groups.
Multi-Release Roadmap
New
“Big Tent” Release Cycle Reform
18
Solving for “How do we allow for the additional projects in the future without breaking down?”
(Current) Tag Categories:
Release Team
Tag Descriptionintegrated-release Frozen tag, not given to new projects. Identifies projects that were integrated prior to Kilo.release: indepdent Projects with this tag “release as needed” and don’t have to coordinate with other projects.
release: at-6mo-cycle-end Projects that commit to being a part of a coordinated release every 6 months. They can still have intermediate releases independent of the 6 month cycle “final” release.
release: has-stable-branches Projects that have stable branches (from the last release in the cycle)release: managed Projects that agree to follow the processes/timelines outlined by the OpenStack Release Management Team
team: diverse-affiliation This tag shows that the developer team for the project is from a diverse set of organizations (1 < 50% and 2 < 80%). This is tested every 6 months.
Details at http://governance.openstack.org/reference/tags
(Current) Tags:
How Do We Know?• Growing skepticism from analysts, reporters, and pundits
• Growing dissatisfaction with certain aspects of OpenStack
• Lots of failures in the field, enough to be worrisome
• Peak OpenStack?
• 6K+ attendees, early signs of slow down in adoption?
• Decide for yourself; I could be calling it early
27
1
2
3
The Growing Skepticism
28
Linthicum believes that despite the fact that OpenStack has "the only game in town" for open source, the implementation hasn't met up to all of the hoopla since
its release. http://searchcloudcomputing.techtarget.com/podcast/OpenStack-talk-of-open-source-town-but-is-it-hype
OpenStack can run a fine private cloud, if you have lots of people to throw at the project and are willing to do lots of coding, according to Alan Waite, a research
director at Gartner.
OpenStack has the following drawbacks as a platform on which to build a private cloud*: 1 Difficulty of implementation 2 Shortage of skills available in the market 3 Conflicting or uncoordinated project governance 4 Weak spots in some projects 5 Integration with existing infrastructure *Recent Q1’2015 Gartner Report
OpenStack Self-Improvement Survey• Intention:
• determine if and where project dissatisfaction exists
• report back to provide perspective on where we need to change
• After 10 days:
• 65+ respondents w/ 30 months average time with OpenStack
• Survey: http://tinyurl.com/improve-openstack [ TAKE ME! ]
29
How Would Your Characterize Your Participation in OpenStack Land?
30
19%
16%
30%
35%
OpenStack DeveloperOpenStack Operator/AdministratorOpenStack End-User (MIA)Pundit, Analyst, Reporter, OpenStack Evangelist, or GroupieOther
Average Time Working With OpenStack: 32 months
What is Your MOST favorite OpenStack Project?
31
NovaSwiftHeat
KeystoneNeutron
IronicCinder
DesignateCeilometer
TroveBarbican
GlanceHorizonManila
OsloSaharaTripleOZaqarOther
0 4 8 12 16Responses
What is Your LEAST favorite OpenStack Project?
32
CeilometerNeutronTripleOCinder
HorizonOslo
GlanceKeystone
NovaHeat
IronicSahara
SwiftTroveZaqar
BarbicanDesignate
ManilaOther
0 4 8 12 16Responses
User Survey Feedback• Neutron:
• “Neutron is a lot more complex and harder to provide real HA” – Survey Respondent
• “Complexity, availability and scalability remain some of the concerns [ of the operators during the Operator Meetup in March ]” – User Survey Team
• Ceilometer: • “adoption has not been rising as quickly as expected … dozens of
comments related to stability and reliability, particularly at scale.” – User Survey Team
33
Well run technology organizations will often throw away or re-architect v1 and even v2 products. Do you think this is a good practice?
35
19%
81%
Yes No
OpenStack Threats• Explosive growth drives complexity
• Continued complexity slows adoption
• Can’t simplify, kill, or re-architect “broken” projects
• Rigid technical governance model (still too centralized)
• Long term vision & product strategy doesn’t emerge (in process)
37
Path to the Plateau of Productivity
40
Plan Item Objective
#1) Streamlining Governance Model empower projects, scale TC, focus Product WG, focus Board and Foundation on marketing and interoperability
#2) Allow Competition force poor projects to evolve or die, allow other projects, particularly non-Python to come under our “big tent”
#3) Conform to Well Known APIs don’t create new APIs in places where they exist (e.g. OAuth 2.0)
#4) Testable Reference Architecturesallow for vertical and horizontal-specific OpenStack
reference implementations and separate infrastructure from platform
#5) Ruthless Simplificationdownloadable “OpenStack Basic IaaS” should be 1-click
download and install to run a POC/trial on a simple stack (1 switch, 10 servers)
41
18 Categories (including retired), 252 Projects
The ASF Scales
Source: http://apache.org/foundation/governance/orgchart
To Manage This… You Need This.
Allow Competing Projects & Multiple Languages
• Competition is good; pretending our shit doesn’t stink is bad • Poor projects must die; survival of the fittest works • There is already leeway for this:
• “Where it makes sense, the project cooperates with existing projects rather than gratuitously competing or reinventing the wheel.”*
• i.e. Competitive projects are OK as long as they have good reason
• Python isn’t good for everything • A bigger tent means allowing non-Python projects • Swift is already experimenting with re-writing pieces in Go Language (golang)
42Source: http://governance.openstack.org/reference/new-projects-requirements.html
–Thierry Carrez, Chairman of the TC, Foundation Release Manager
“OpenStack is about community, common values, and a common governance model.”
43
Keystone API• Seriously … WTF?
• There are dozens of well known, documented, scalable, tested, standard APIs for authN/authZ
• OAuth1/2, SAML, Kerberos
• There is no excuse for creating something from whole cloth
• Google is secure as hell and they use OAuth 2.0
• You aren’t better at security than the Google team; sorry
• We don’t apply this standard to our community (completely new Nova API anyone?)
44
Example Reference Architectures
45
OpenStack Interop Standard RA “Key” Components RA Optional Components
Basic IaaS 1+ of Nova/Magnum/Ironic OAuth 2.0 Server (Keystone or other) Glance, Horizon
Advanced IaaSOpenStack Basic IaaS Cinder, Swift Neutron (or an alternative?) OAuth 2.0 Server (Keystone or other)
Glance, Horizon
OpenStack AppServices Zaqar, Trove, Designate, OAuth2.0 Horizon
OpenStack AppManagement Heat, Murano, Mistral, Horizon, OAuth2.0 Horizon
OpenStack for NFVBasic IaaS Pluggable SDN Controller w/ Neutron APIs
OpenStack Public Cloud Advanced IaaS + OpenStack App Svcs + OpenStack App Mgmt ec2api, gce-api, etc.
P2 tests
How it Might Work
46
Reference Architecture
Default Config Opts
Key + Optional Projects { Project 1
Project 2
Interoperability Test Suite
Defined “Capabilities” (previously “DefCore”)
RefStack
P1 tests
Capabilities Tests API Code
Owner(s): Infrastructure Team & Working Groups
TC, Board, Vertical/Horizontal Working Groups, Community, &
Foundation
PTLs & key committers/reviewers (more like Apache PMC??)
Unit Tests
We Are t3h Borg. You Will Be Assimilated.
47
“Maybe I’m an idiot, but I have no idea what anyone is talking about. What is it? It’s complete gibberish. It’s insane. When is this idiocy
going to stop?” Larry Ellison on Cloud
ComputerWorld, July 2000
"This 'telephone' has too many shortcomings to be seriously considered as a means of
communication. The device is inherently of no value to us.”, Western Union internal memo, 1876.
Decca Records rejected the Beatles, saying "guitar groups are on the way out" and "The Beatles have no future in show business,"
We Can Do It!• Interrelated, but not interdependent projects
• Testable reference architectures that are interoperable
• Streamline governance
• Survival of the fittest project and programming language
• OpenStack is not specific code or APIs, it’s:
• Community, common values, and common governance
48