Post on 27-Jan-2015
description
Chef 11 Preview/Chef for OpenStack
Boston 1/22/2013
Matt Raymatt@opscode.com
@mattray
Tuesday, January 22, 13
What's New Since Chef 0.10?
• Windows support
• Librarian
• Spiceweasel
• Foodcritic
• Cookbooks in separate repositories
• Private Chef
• Food Fight podcast
• knife plugins for openstack, hp, azure, gce
• 0.10 renumbered to Chef 10
• full-stack client
• why-run
• output formatters
• Test Kitchen
• Solaris support
• Chef Developer Summit(2!)
• Berkshelf
• ChefConf
• Fauxhai
• docs.opscode.com
• Hangouts for reviews
• nyan-cat output formatter
Tuesday, January 22, 13
Vaguely chronological, so much good stuff in the Chef Community
What is NOT in Chef 11
• CouchDB
• Ruby-based chef-server-api
• depsolver, gecode, treetop
• Merb
• OpenID support in Web UI
• Migration tooling (coming soon)
http://trainweb.org/carl/Pullman150/IMG_3380.jpgTuesday, January 22, 13
What’s NEW in Chef 11
• PostgreSQL
• erchef
• nginx
• Rails
• bookshelf
• omnibus-chef server
• chef-apply
• partial search
• users with key pairs
• partials in templates
• knife-essentials
Tuesday, January 22, 13
nginx
erchef
solr(lucene)
bookshelf
PostgreSQLfilesystemcookbookstore
filesystemsearchindex
RabbitMQ
chef-expander
chef-webui
Chef Server Architecture
Tuesday, January 22, 13
Postgres has replaced Couch (who knew SQL was so good?)Erlang-based erchef has replaced Ruby-based chef-server-apichef-webui has been upgraded to Rails 3 from MerbBookshelf is our S3-compatibilish storage service
erchef
chef_wm
chef_index chef_dbchef_authnchef_objects
solrbookshelfPostgreSQL
RabbitMQ
erchef Architecture
Tuesday, January 22, 13
Erlang applications, each on GitHub
CPU Usage on Chef Server
Tuesday, January 22, 13
These are old graphs from last year. Code is actually much faster and more efficient now that we've had a year of tuning.This is with erchef and Ruby code on the same box3 CPU vm graph is aggregate
CouchDB Uptime
Tuesday, January 22, 13
the smaller spikes were when we chron'd restarts
Database CPU
CouchDB MySQL
Tuesday, January 22, 13
Postgres is about the same performance, but we've found we're able to focus optimizations on a single rdbms and put time that we would otherwise have to spend on ensuring both backend dbs work properly into other areas of the code for fixes, enhances, features.
Database Memory
CouchDB MySQL
Tuesday, January 22, 13
Nice and flat
Database Load Average
CouchDB MySQL
Tuesday, January 22, 13
Licensing is much better for us with Postgres
API Average Latency
Tuesday, January 22, 13
And things have gotten better from here with Private Chef and Open Source Chef. Not everything has been ported to Opscode Hosted Chef yet. Work has started to complete the migration of OHC to the OPC code base, things with OHC will get far better.
omnibus-chef server
• full-stack for the Chef server, everything you need
• installs to /opt/chef-server
• RPMs, DEBs for now
• build your own packages
• chef-server-ctl
• status
• start/stop
• tail
http://apod.nasa.gov/apod/astropix.html
Tuesday, January 22, 13
Much of the same tooling OPC
chef-apply
• run a single recipe file
• without modifying the node's run_list
• included with Chef gem, /usr/bin/chef-apply
• chef-apply /path/to/recipe_file
• chef-apply "content of a recipe file"
http://www.flickr.com/photos/albill/sets/72157628046395000/
Tuesday, January 22, 13
http://tickets.opscode.com/browse/CHEF-3571
partial search
http://www.flickr.com/photos/albill/sets/72157628046395000/
partial_search(:node, 'role:web', :keys => { 'name' => [ 'name' ], 'ip' => [ 'ipaddress' ], 'kernel_version' => [ 'kernel', 'version' ] }).each do |result| puts result['name'] puts result['ip'] puts result['kernel_version']end
• instead of entire node, just the pieces you want
• massive reduction in bandwidth and memory
• http://community.opscode.com/cookbooks/partial_search
• to use with Chef 10 and Hosted Chef today
Tuesday, January 22, 13
users with keypairs
• users can have key pairs, just like clients
• knife actions as a user instead of a client
• post a public key when you create a user/client
http://www.flickr.com/photos/albill/sets/72157628046395000/
Tuesday, January 22, 13
For users in Chef 11, the key point is:Before: users were just a concept of the webui. If you wanted a knife setup, you needed a client.After: users can have key pairs just like clients. So you can do knife actions as your user.
<?xml version='1.0' encoding='UTF-8'?>
<server xmlns="urn:jboss:domain:1.2"> <extensions> <% if @infinispan -%> <extension module="org.jboss.as.clustering.infinispan"/> <% end -%> <extension module="org.jboss.as.web"/> <% if @webservices -%> <extension module="org.jboss.as.webservices"/> <% end -%> <extension module="org.jboss.as.weld"/> </extensions> <% if @infinispan include_template "infinispan.xml.erb" end -%> <% if @webservices include_template "webservices.xml.erb" end -%> <socket-binding-group name="standard-sockets" > ...... </socket-binding-group></server>
• Thanks to Andrea Campi!
partials in templates
http://www.flickr.com/photos/modern_fred/2095565021/
Tuesday, January 22, 13
knife-essentials
• Unified commands that work on everything
• https://github.com/jkeiser/knife-essentials
• knife download [pattern1 pattern2 ...]
• knife download roles data_bags cookbooks/emacs
• knife diff cookbooks/*apache*
• knife show *base*
• knife upload apache*
• knife list data_bags/users
• knife deps roles/base.json
http://photography.nationalgeographic.com/photography/photo-of-the-day/Tuesday, January 22, 13
knife downloadknife diffknife showknife listknife upload
Chef 11 Breaking Changes
• http://wiki.opscode.com/display/chef/Breaking+Changes+in+Chef+11
• chef-shell (formerly shef)
• no implicits node attributes (no more node['blah'] = 'foo')
• attribute files may access role and environment attributes
• delayed notifications run after failed converge
• encrypted data bag item format change
• chef-client lock so safe from simultaneous runs
http://www.flickr.com/photos/modern_fred/2096352938/Tuesday, January 22, 13
More changes on the wiki page
Chef 11 Server Preview
• http://wiki.opscode.com/display/chef/Chef+11+Server+Preview
• Nightly Builds
• Centos 5 & 6
• Ubuntu 10.04, 11.04 and 12.04
• Everything is on GitHub
• More testing means better releases!
• Client is in beta: http://lists.opscode.com/sympa/arc/chef-dev/2013-01/msg00028.html
http://www.brandonbird.com/lazysunday.html
Tuesday, January 22, 13
Coming soon!
Overview & Status of Chef for OpenStack
Tuesday, January 22, 13
Chef for OpenStack: Who
Tuesday, January 22, 13
These companies are currently involved to some extent
Chef for OpenStack: Why
• Community for the automated deployment and management of OpenStack
• Reduce fragmentation and encourage collaboration
• Deploying OpenStack is not "secret sauce"
• Project not a product
• Apache 2 license
Tuesday, January 22, 13
Chef for OpenStack: What
• Chef Repository for Deploying OpenStack
• Documentation for Chef for OpenStack
• Cookbooks• Keystone
• Glance
• Nova
• Horizon
• Swift
• Quantum
• Cinder
• knife-openstackTuesday, January 22, 13
Chef for OpenStack: Where
• groups.google.com/group/opscode-chef-openstack
• #openstack-chef on irc.freenode.net
• github.com/opscode/openstack-chef-repo
• github.com/mattray/openstack-chef-docs
• github.com/opscode-cookbooks/
• keystone, glance, nova, horizon, swift,quantum,cinder
• github.com/opscode/knife-openstack
• @chefopenstack
Tuesday, January 22, 13
• Chef repo for Essex (2012.1.1 tag)
• Operating Systems (Ubuntu 12.04)
• Hypervisors (KVM, LXC)
• Databases (MySQL)
• FlatDHCP & VLAN networking
• manual floating IPs documented
• Test Kitchen integration
Chef for OpenStack: When (Today)
Tuesday, January 22, 13
Chef for OpenStack: When (Tomorrow)
• Documentation (docs.opscode.com)
• Folsom is under active development
• AT&T, DreamHost and Rackspace have active branches
• Move to openstack-common
• Cinder (lvm, Netapp)
• Quantum (Nicira with Open vSwitch)
Tuesday, January 22, 13
• Documentation (docs.opscode.com)
• Continuous Integration testing
• Grizzly & trunk(!?)
• Cinder (Ceph)
• Quantum (Midokura)
• Hypervisors (Hyper-V, bare metal)
• Databases (PostgreSQL)
• Operating Systems (RHEL, Debian, SUSE)
• HA Configurations
Chef for OpenStack When: (Roadmap)
Tuesday, January 22, 13
github.com/mattray/openstack-chef-docsmoving to
github.com/opscode/chef-docs
Chef for OpenStack: How
Tuesday, January 22, 13
Working on a permanent URLHTML, PDF, Epub for your iPad
Questions?
Boston 1/22/2013
Matt Raymatt@opscode.com
@mattray
Tuesday, January 22, 13
ChefConf 2013 April 2013 "OPSCODE-MEETUP"Metarepo and CI work