Infrastructure as Software - PuppetConf 2014

INFRASTRUCTURE AS SOFTWAREINFRASTRUCTURE AS SOFTWAREDustin J. Mitchell

[email protected] 24, 2014

Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...

1 of 37 09/24/2014 03:32 PM

CODECODEInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...

2 of 37 09/24/2014 03:32 PM

CODE = SNIPPETCODE = SNIPPET

var width=150; // width of the eyes in pixelsvar colour="#06f"; // colour of the eye - bluey green in this casevar iris="#000"; // colour of the iris (normally black);/***************************\* Moving Eyeballs Effect **(c)2012-3 mf2fm web-design ** http://www.mf2fm.com/rv ** DON'T EDIT BELOW THIS BOX *\***************************/var swide=800;function addLoadEvent(funky) { var oldonload=window.onload; if (typeof(oldonload)!='function') window.onload=funky; else window.onload=function() {...


3 of 37 09/24/2014 03:32 PM

CODE = SNIPPETCODE = SNIPPET

# == Class: baseconfig## Performs initial configuration tasks for all Vagrant boxes.#class baseconfig { exec { 'apt-get update': command => '/usr/bin/apt-get update'; }

host { 'hostmachine': ip => '192.168.0.1'; }

file { '/home/vagrant/.bashrc': owner => 'vagrant', group => 'vagrant', mode => '0644', source => 'puppet:///modules/baseconfig/bashrc'; }}


4 of 37 09/24/2014 03:32 PM

CODE = SCRIPTCODE = SCRIPT

#! /usr/bin/env python

class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):

def do_POST(self): content_length = int(self.headers['content-length']) data = self.rfile.read(content_length) self.send_response(200) self.end_headers() self.wfile.write("accepted.\n") self.wfile.close() now = int(time.time()) filename = os.path.join(REPORT_DIR, "report-%s.yaml" % (now,)) fd = os.open(filename, os.O_EXCL|os.O_CREAT|os.O_WRONLY) os.fdopen(fd, "w").write(data)

def main(): logging.basicConfig(format="%(asctime)s %(message)s", level=logging.DEBUG) SocketServer.TCPServer.allow_reuse_address = True httpd = SocketServer.TCPServer(("", PORT), RequestHandler) httpd.serve_forever()

main()


5 of 37 09/24/2014 03:32 PM

--

“My friend Clift Norris has identified a fundamental constantthat I call Norris’ number, the average amount of code an

untrained programmer can write before he or she hits a wall. Cliftestimates this as 1,500 lines. Beyond that the code becomes so

tangled that the author cannot debug or modify it withoutherculean effort.”

John D. Cook


6 of 37 09/24/2014 03:32 PM

SOFTWARE IN PUPPET?SOFTWARE IN PUPPET?Let me tell you a story..


7 of 37 09/24/2014 03:32 PM

tinyurl.com/puppetagainMEET PUPPETAGAINMEET PUPPETAGAINOpen SourceWritten in PuppetCross-PlatformHighly AvailableSecureDesigned to Manage Job-Runners


8 of 37 09/24/2014 03:32 PM

SOFTWARESOFTWAREArchitectural models


9 of 37 09/24/2014 03:32 PM

OPENSTACK ARCHITECTUREOPENSTACK ARCHITECTUREInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...

10 of 37 09/24/2014 03:32 PM

APPROACHABILITYAPPROACHABILITYIf I change this, what will happen?

Where should I start reading code?


11 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: TOPLEVELPUPPETAGAIN: TOPLEVELclass toplevel::base { include users::root}class toplevel::server inherits toplevel::base { include puppet::periodic include cron}class toplevel::server::mozpool inherits toplevel::server { include bmm include mozpool}


12 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: TOPLEVELPUPPETAGAIN: TOPLEVELnode "mobile-imaging1.p1.releng.scl3.mozilla.com" { include toplevel::server::mozpool}


13 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: CONFIGURATIONPUPPETAGAIN: CONFIGURATION# modules/config/manifests/base.ppclass config::base { $ntp_server = ''}# manifests/moco-config.ppclass config inherits config::base { $ntp_server = 'time.mozilla.org'}


14 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: CONFIGURATIONPUPPETAGAIN: CONFIGURATIONclass ntp::config { include ::config if ($::config::ntp_server) { .. }}


15 of 37 09/24/2014 03:32 PM

SOFTWARESOFTWAREArchitectural modelsControlled Interdependencies


16 of 37 09/24/2014 03:32 PM

APACHE: MODULESAPACHE: MODULESInfrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...

17 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: DIRSPUPPETAGAIN: DIRSclass dirs::builds { file { "/builds": ensure => directory; }}class dirs::builds::slave { include dirs::builds file { "/builds/slave": ensure => directory; }}


18 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: DIRSPUPPETAGAIN: DIRSclass talos { include dirs::builds::slave file { "/builds/slave/talos-slave": ensure => directory; }}


19 of 37 09/24/2014 03:32 PM

SOFTWARESOFTWAREArchitectural modelsControlled InterdependenciesOrganizing Principles


20 of 37 09/24/2014 03:32 PM

PA: PRINCIPLE OF LEAST SURPRISEPA: PRINCIPLE OF LEAST SURPRISE

.. installs mig-agent .. or dies trying

include mig_agent::install


21 of 37 09/24/2014 03:32 PM

PA: PRINCIPLE OF LEAST SURPRISEPA: PRINCIPLE OF LEAST SURPRISEclass mig_agent::install { case $operatingsystem { CentOS: { .. } default: { fail("Cannot install on $operatingsystem") } }}


22 of 37 09/24/2014 03:32 PM

SOFTWARESOFTWAREArchitectural modelsControlled InterdependenciesOrganizing PrinciplesAbstractions


23 of 37 09/24/2014 03:32 PM

PUPPET: PACKAGESPUPPET: PACKAGES

Write once, run everywhere, right?

package { 'httpd': ensure => '2.2.15';}


24 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: PACKAGESPUPPETAGAIN: PACKAGESinclude packages::httpd


25 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: PACKAGESPUPPETAGAIN: PACKAGESclass packages::httpd { case $::operatingsystem { CentOS: { package { "httpd": ensure => latest; } } Ubuntu: { package { "apache2": ensure => latest; } } Darwin: { # installed by default } default: { fail("cannot install on $::operatingsystem") } } }


26 of 37 09/24/2014 03:32 PM

WRITING SOFTWAREWRITING SOFTWAREIS HARDIS HARD


27 of 37 09/24/2014 03:32 PM

CHALLENGESCHALLENGESPackage repositories are part of the code


28 of 37 09/24/2014 03:32 PM

REPOSITORIES:REPOSITORIES:PINNING IS HARDPINNING IS HARD

Un-specified prerequisite packages aren't pinnedCan confuse package managersModifying the repo makes production changes


29 of 37 09/24/2014 03:32 PM

REPOSITORIES:REPOSITORIES:MIRRORING IS HARDMIRRORING IS HARD

“Can we update mirrors now?”

“No.”


30 of 37 09/24/2014 03:32 PM

REPOSITORIES:REPOSITORIES:SHARING IS HARDSHARING IS HARD


31 of 37 09/24/2014 03:32 PM

REPOSITORIES:REPOSITORIES:THEY'RE HUGETHEY'RE HUGE

[[email protected] dmitchell]# df -h /dataFilesystem Size Used Avail Use% Mounted on/dev/mapper/vg_relengpuppet2-lv_data 414G 315G 78G 81% /data


32 of 37 09/24/2014 03:32 PM

CHALLENGESCHALLENGESPackage repositories are part of the codeInteractions are hard to model


33 of 37 09/24/2014 03:32 PM

MODULE INTERACTIONSMODULE INTERACTIONS'httpd' module installs Apache'rsyslog' module installs rsyslogWhen both are installed, we want to send access logs to rsyslog


34 of 37 09/24/2014 03:32 PM

MODULE INTERACTIONSMODULE INTERACTIONS

Httpd has to know about rsyslog?

class httpd::logging { include rsyslog::config_dir file { "${rsyslog::config_dir::dir}/httpd.conf": content => template("${module_name}/rsyslogd_httpd.conf.erb"); }}


35 of 37 09/24/2014 03:32 PM

CHALLENGESCHALLENGESPackage repositories are part of the codeInteractions are hard to modelAcceptance-level testing is hard


36 of 37 09/24/2014 03:32 PM

GO FORTH ANDGO FORTH ANDWRITE SOFTWAREWRITE SOFTWARE


37 of 37 09/24/2014 03:32 PM

Infrastructure as Software - PuppetConf 2014

Technology

Transcript of Infrastructure as Software - PuppetConf 2014