Custom Live Media Spinning

Jon Cowie, Etsy.comjcowie@etsy.com

@jonlives

Thursday, 11 July 13

Thursday, 11 July 13

1.5 billion page views$117 million of goods sold

950 thousand users

(in december ‘12)

Thursday, 11 July 13

~800 physical hosts~300 kvm & lxc VMs

CentOS 5 & 6

Thursday, 11 July 13

RacktablesCobbler

Chef

Thursday, 11 July 13

Old Host Build Process

Configure StorageConfigureSwitchports

Configure DNSConfigure Cobbler

Power on HostProfit!

Rack Server Configure ILO

Thursday, 11 July 13

New Host Build Process

Configure StorageConfigureSwitchports

Configure DNSConfigure Cobbler

Power on HostProfit!

Rack Server Configure ILO

Gabriel

Thursday, 11 July 13

New Host Build Process

Configure StorageConfigureSwitchports

Configure DNSConfigure Cobbler

Power on HostProfit!

Rack Server Configure ILO

Sledgehammer

Thursday, 11 July 13

Sledgehammer

Runs before OS InstallConfigures ILO

Configures RAID & GPT

Thursday, 11 July 13

Live CD!

Memory residentNetwork bootable*Easily customisable

Thursday, 11 July 13

Sledgehammer Live CD

Minimal installNetwork, disk tools

Ruby & GemsSledgehammer “Runner”

Thursday, 11 July 13

Sledgehammer Runner

Mount NFSDownload config

Download payload RPMRun payload

Thursday, 11 July 13

Sledgehammer Payload

Configure ILO Set up RAID

Configure GPT Burn-in tests

Update Racktables

Thursday, 11 July 13

So how does it work?

Thursday, 11 July 13

LiveCD Primer

-isolinux-initrd0.img-vmlinuz0

-LiveOS-squashfs.img-ext3fs.img

Thursday, 11 July 13

LiveCD Primer

Load kernel and ramdiskMount ro / fs

Create rw overlayRun init

Thursday, 11 July 13

First modification!

Overlay is fixed size.Copy entire FS to tmpfs

Mount rw / FS

Thursday, 11 July 13

Second modification!

Custom payloadMust run automatically...

Thursday, 11 July 13

/etc/init/start-ttys.conf

...env  ACTIVE_CONSOLES=/dev/tty[1-­‐6]...for  tty  in  $(echo  $ACTIVE_CONSOLES)  ;  do      ...      initctl  start  tty  TTY=$ttydone...

Thursday, 11 July 13

/etc/init/start-ttys.conf...env  ACTIVE_CONSOLES=/dev/tty[1-­‐6]...for  tty  in  $(echo  $ACTIVE_CONSOLES)  ;  do      ...

if  [  \"\$tty\"  ==  \"\/dev/tty1\"  ]    then          initctl  start  centosdojo  TTY=\$tty    else          initctl  start  tty  TTY=\$tty    fidone...

Thursday, 11 July 13

/etc/init/centosdojo.conf

stop  on  runlevel  [012456]exec  /usr/bin/openvt  -­‐c  1  -­‐w  -­‐f  -­‐-­‐  /opt/centosdojo/runner.rb

Thursday, 11 July 13

So how do I build it?

Thursday, 11 July 13

http://tiny.cc/dojo2013

Thursday, 11 July 13