Puppet at GitHub - PuppetConf 2013
-
Upload
puppet-labs -
Category
Technology
-
view
2.835 -
download
2
description
Transcript of Puppet at GitHub - PuppetConf 2013
Puppet at GitHub
@wfa r rGitHub
Operations
Known Aliases:
King of Kebabs
The Chairman
Mr. Caremad !
The State of Puppet at GitHub
"
" The State of Puppet at GitHub
github/boxen
" The State of Puppet at GitHub
~1.5 years old
" The State of Puppet at GitHub
open-sourced ~7 months ago
" The State of Puppet at GitHub
~240 open-source puppet modules
" The State of Puppet at GitHub
puppet 3.latest
" The State of Puppet at GitHub
supports ruby 1.8.7, 1.9.3, 2.0.0
" The State of Puppet at GitHub
hiera
" The State of Puppet at GitHub
OS X
" The State of Puppet at GitHub
Linux support in-progress
" The State of Puppet at GitHub
#tomorrow @ 2:20pm
" The State of Puppet at GitHub
github/puppet
" The State of Puppet at GitHub
$
" The State of Puppet at GitHub
~5 years old
" The State of Puppet at GitHub
0.24.x — 2.7.x
" The State of Puppet at GitHub
121 modules
" The State of Puppet at GitHub
~280k lines of code
" The State of Puppet at GitHub
every single employee has commit access
" The State of Puppet at GitHub
15.5k commits to master past 12 months
" The State of Puppet at GitHub
by 86 contributors
" The State of Puppet at GitHub
% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %
% %
" The State of Puppet at GitHub
% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %
% %
42.5%
" The State of Puppet at GitHub
% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %% % % % % % % % % % % % % % % % % % % %
% %
now with ops taken out
" The State of Puppet at GitHub
0
100
200
300
40020
12-0
8-25
2012
-10-2
0
2012
-12-15
2013
-02-
09
2013
-04-
06
2013
-06-
01
2013
-07-
27
commits on master per week, last 12 months
" The State of Puppet at GitHub
0
1000
2000
3000
4000total commits by author past year
" The State of Puppet at GitHub
0
175
350
525
700total commits by author past year, except ops
" The State of Puppet at GitHub
0
175
350
525
700total commits by authors with >10 commits, past year, except ops
" The State of Puppet at GitHub
single puppetmaster
" The State of Puppet at GitHub
rubygems
" The State of Puppet at GitHub
ruby 1.8.7
" The State of Puppet at GitHub
unicorn
" The State of Puppet at GitHub
puppet 2.7.latest
" The State of Puppet at GitHub
~600 nodes
" The State of Puppet at GitHub
run hourly via crond
" The State of Puppet at GitHub
puppetdb
" The State of Puppet at GitHub
nagiosdb
" The State of Puppet at GitHub
"yo puppetdb, gimme all your nagios::object::* resources so I can
update this nagios config"
" The State of Puppet at GitHub
filtergendb
" The State of Puppet at GitHub
"yo puppetdb, gimme all the filtergen::rule resources I would realize
so I can update this filtergen config"
" The State of Puppet at GitHub
⚡ puppetdb ⚡
" The State of Puppet at GitHub
as it turns out, an api call is faster than running puppet on a host
" The State of Puppet at GitHub
gpanel
" The State of Puppet at GitHub
"Imagine Puppet Dashboard meets Razor and went on a weekend trip to the
beach with the Heroku API and drank epic amounts of blue drink"
" The State of Puppet at GitHub
aka we reinvented our own, smaller version of Foreman
" The State of Puppet at GitHub
inventory
" The State of Puppet at GitHub
" The State of Puppet at GitHub
" The State of Puppet at GitHub
app configuration
" The State of Puppet at GitHub
" The State of Puppet at GitHub
versioning of configuration values
" The State of Puppet at GitHub
Create nil => 1Update 1 => 2Delete 2 => nil
" The State of Puppet at GitHub
let's make credential rolling less awful
" The State of Puppet at GitHub
provisioning
" The State of Puppet at GitHub
provisioning is typically awful
" The State of Puppet at GitHub
we sprinkled in some ChatOps
" The State of Puppet at GitHub
" The State of Puppet at GitHub
a little bit later...
" The State of Puppet at GitHub
<MACHINE>
" The State of Puppet at GitHub
enc
" The State of Puppet at GitHub
we have the dumbest ENC out there
" The State of Puppet at GitHub
⋆
" The State of Puppet at GitHub
# /usr/local/sbin/fetch_gpanel_enc \ fe1.rs.github.com
---parameters: gpanel_cabinet: D20-13 gpanel_enabled: true gpanel_monitored: false
" The State of Puppet at GitHub
we never specify classes via the ENC
" The State of Puppet at GitHub
any variables we pass through are prefixed with gpanel_
How GitHub writes Puppet
(
( How GitHub Writes Puppet
rodjek/puppet-lint
( How GitHub Writes Puppet
( How GitHub Writes Puppet
if you aren't using puppet-lint to audit your puppet codebase,
you are doing it wrong
( How GitHub Writes Puppet
puppet-lint enforces the Puppet Labs style guide
( How GitHub Writes Puppet
puppet-lint can even fix a ton of linter errors for you
( How GitHub Writes Puppet
put it in a pre-commit hook
( How GitHub Writes Puppet
$ git commit -am "can't lint this"
modules/github/manifests/role/redis.pp: syntax okmodules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118
1 errors found, aborting commit.
( How GitHub Writes Puppet
and then buy rodjek a beer
( How GitHub Writes Puppet
rodjek/rspec-puppet
( How GitHub Writes Puppet
( How GitHub Writes Puppet
if you aren't writing tests for your puppet code before
running it on a server,you are doing it wrong
( How GitHub Writes Puppet
use whatever framework/library
( How GitHub Writes Puppet
rspec-puppet just happens to be a pretty good one
( How GitHub Writes Puppet
put it in a pre-commit hook
( How GitHub Writes Puppet
$ git commit -am "tests dont pass but whatever lol"
1) Expected redis::server would include class "more_than_a_single_c_thread"
1 failures encountered, aborting commit.
( How GitHub Writes Puppet
and then buy rodjek another beer
( How GitHub Writes Puppet
node definitions
( How GitHub Writes Puppet
we don't use an ENC to describe node classes
( How GitHub Writes Puppet
node /^github-redis\d+/ { class { 'github::role::polling_redis': enabled => $::gpanel_enabled, environment => $::gpanel_environment, private_ipv4 => $::ipaddress, }}
( How GitHub Writes Puppet
abstractions all the way down
( How GitHub Writes Puppet
treat your site classes as cascades down to your dist classes
( How GitHub Writes Puppet
class redis::server( # params) {
class { 'redis::config': ... } -> class { 'redis::package': ... } ~> class { 'redis::service': ... }
}
( How GitHub Writes Puppet
class github::redis( # params) inherits github::defaults {
$memory = $environment ? { 'stg' => '2G', default => $half_memory_gb }
class { 'redis::server': ... }
}
( How GitHub Writes Puppet
class github::role::polling_redis( # params) {
class { 'github::redis': # overrides based on specific node }
}
( How GitHub Writes Puppet
augeas
( How GitHub Writes Puppet
you know what's not awesome?
( How GitHub Writes Puppet
an erb template that requires your class to take 52876423 parameters so you can configure every possible
value in my.cnf
( How GitHub Writes Puppet
augeas { 'my.cnf/performance': context => '/files/etc/mysql/my.cnf/mysqld', changes => [ # automatic dump/restore 'set innodb_auto_lru_dump 18000',
# innodb "set innodb_buffer_pool_size ${innodb_buffer_pool_size}", 'set innodb_log_file_size 256M', 'set innodb_log_buffer_size 8M', 'set innodb_lazy_drop_table 1', ],require => Percona::Server[$::fqdn]
}
( How GitHub Writes Puppet
it can seem complex and scary
( How GitHub Writes Puppet
it is an amazing tool
How GitHub ships Puppet
)
) How GitHub Ships Puppet
continuous integration
) How GitHub Ships Puppet
commit gets pushed
) How GitHub Ships Puppet
jenkins runs the test suite
) How GitHub Ships Puppet
status gets posted back to GitHub.com
) How GitHub Ships Puppet
tmm1/test-queue
) How GitHub Ships Puppet
) How GitHub Ships Puppet
remembers how to better parallelize tests
) How GitHub Ships Puppet
continuous deployment
) How GitHub Ships Puppet
tests passed for commit, Hubot auto-deploys
) How GitHub Ships Puppet
branch deploy everything
) How GitHub Ships Puppet
a lot of shops have a few environments
) How GitHub Ships Puppet
testing
staging
production
) How GitHub Ships Puppet
currently we have 181 environments
) How GitHub Ships Puppet
that's not the same as 181 nodesnot running production
) How GitHub Ships Puppet
we only have 5 of those
) How GitHub Ships Puppet
Hubot automatically merges the master branch before allowing any
branch to deploy
) How GitHub Ships Puppet
ChatOps
) How GitHub Ships Puppet
202 employees100% access to Puppet
100% trust
) How GitHub Ships Puppet
the list of roles you'd never think would touch production puppet
webkit developercore git developers
core ruby developerssvn developers
search developerfrontend designers
windows developersmac developers
supportenterprise sales
) How GitHub Ships Puppet
every puppet run happens in chat
) How GitHub Ships Puppet
all puppet output goes to chat
) How GitHub Ships Puppet
everyone can see everything
) How GitHub Ships Puppet
everyone can do anything
) How GitHub Ships Puppet
hands-on learning, by accident
) How GitHub Ships Puppet
eliminate disruptive questions
The Future of Puppet at GitHub
*
* The Future of Puppet at GitHub
puppet 3.x
* The Future of Puppet at GitHub
upgrading a large, old codebase from 2.7 to 3 is really painful
* The Future of Puppet at GitHub
we've been talking about it for a year
* The Future of Puppet at GitHub
we're still trying to push forward, but...
* The Future of Puppet at GitHub
helping maintain security fixes for 2.7
* The Future of Puppet at GitHub
goal is to be on 3.2 by end of year
* The Future of Puppet at GitHub
ruby 2.0
* The Future of Puppet at GitHub
moar faster rubby
* The Future of Puppet at GitHub
we get this for free when we move to 3
* The Future of Puppet at GitHub
mcollective
* The Future of Puppet at GitHub
aka the story of github/shell and the wonders of ssh in a for-loop
* The Future of Puppet at GitHub
there are a lot of cool things about mcollective
* The Future of Puppet at GitHub
I think it will be a part of our stack in the future
* The Future of Puppet at GitHub
but you can still go pretty damn far with bash and ssh
* The Future of Puppet at GitHub
even more puppetdb tooling
* The Future of Puppet at GitHub
waiting for multiple runs to converge exported resources is painful
* The Future of Puppet at GitHub
use the puppetdb API to skip all that
* The Future of Puppet at GitHub
even more gpanel integration
* The Future of Puppet at GitHub
"databags"
* The Future of Puppet at GitHub
"node search"
* The Future of Puppet at GitHub
steal good ideas from other tools
closing thoughts
"all software is terrible"
— anyone who's worked with software long enough
"no software is better than no software"— rtomayko
the plumbing doesn't matter when all you care about is the porcelain
write some damned good porcelain
THIS IS RODJEK
HE'S NORMALLY NOT THIS BLURRY
HE IS ALSO NOT MY BROTHER, BUT YOU REALLY SHOULD BUY
HIM A BEER
YOU SHOULD BUY THIS MAN A LOT OF BEER
THANKS
https://speakerdeck.com/wfarr/
puppet-at-github-puppetconf-2013
!