Puppet at the centre of everything
-
Upload
server-density -
Category
Technology
-
view
301 -
download
2
description
Transcript of Puppet at the centre of everything
![Page 1: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/1.jpg)
Puppet at the centre of everything
![Page 2: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/2.jpg)
David Mytton
Woop Japan!
![Page 3: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/3.jpg)
![Page 4: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/4.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
![Page 5: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/5.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•50% virtual
![Page 6: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/6.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•50% virtual
![Page 7: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/7.jpg)
Server Density Architecture
•100 servers - Ubuntu 12.04
•Nginx, Python, MongoDB
•25TB data per month
•50% virtual
![Page 8: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/8.jpg)
Use cases
• Config
![Page 9: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/9.jpg)
Use cases
• Config
• Failover
![Page 10: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/10.jpg)
Use cases
• Config
• Code deploys
• Failover
![Page 11: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/11.jpg)
Use cases
• Config
• Code deploys
• System updates
• Failover
![Page 12: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/12.jpg)
• Base manifest
Config
![Page 13: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/13.jpg)
![Page 14: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/14.jpg)
![Page 15: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/15.jpg)
• Base manifest
Config
• Sub manifests
![Page 16: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/16.jpg)
![Page 17: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/17.jpg)
• base
• softlayer
• softlayer-dedicated
• rackspace
• ...
![Page 18: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/18.jpg)
Failover
• /etc/hosts
<%= metrics2IP %> alerts.honshuu.prod audit.honshuu.prod auth.honshuu.prod cloud.honshuu.prod datastore.honshuu.prod inventory.honshuu.prod ...
![Page 19: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/19.jpg)
![Page 20: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/20.jpg)
![Page 21: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/21.jpg)
![Page 22: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/22.jpg)
![Page 23: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/23.jpg)
Failover
• /etc/hosts
108.168.254.33 puppet
![Page 24: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/24.jpg)
![Page 25: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/25.jpg)
Expected
Dealing with outages
![Page 26: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/26.jpg)
Really test your vendors
Dealing with outages
![Page 27: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/27.jpg)
Simulations
Dealing with outages
![Page 28: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/28.jpg)
• Commit
Code deploys
![Page 29: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/29.jpg)
![Page 30: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/30.jpg)
• Commit
Code deploys
• Build
![Page 31: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/31.jpg)
![Page 32: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/32.jpg)
• Commit
Code deploys
• Build
• Deploy
![Page 33: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/33.jpg)
![Page 34: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/34.jpg)
![Page 35: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/35.jpg)
![Page 36: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/36.jpg)
![Page 37: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/37.jpg)
/opt/puppet/bin/mco puppetd runonce -I <host>
![Page 38: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/38.jpg)
• Commit
Code deploys
• Build
• Deploy
• Puppet run
![Page 39: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/39.jpg)
![Page 40: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/40.jpg)
Code deploys
Advantages
APIs
![Page 41: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/41.jpg)
Code deploys
Advantages
APIs
Already there
![Page 42: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/42.jpg)
Code deploys
Advantages
APIs
Already there
System facts
![Page 43: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/43.jpg)
Code deploys
Disadvantages
Slow
![Page 44: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/44.jpg)
Code deploys
Disadvantages
Slow
Puppet run already
![Page 45: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/45.jpg)
Code deploys
Disadvantages
Slow
Puppet run already
Eventually consistent
![Page 46: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/46.jpg)
System updates
• Bash script
![Page 47: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/47.jpg)
System updates
• Bash script
• Remote commands
![Page 48: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/48.jpg)
![Page 49: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/49.jpg)
System updates
• Bash script
• Remote commands
• Notifications
![Page 50: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/50.jpg)
System updates
• Bash script
• Remote commands
• Notifications
• Canaries
![Page 51: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/51.jpg)
![Page 52: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/52.jpg)
•Build master (buildbot): VM x2 CPU 2.0Ghz, 2GB RAM – $89/m
•Build slave (buildbot): VM x1 CPU 2.0Ghz, 1GB RAM – $40/m
•Staging load balancer: VM x1 CPU 2.0Ghz, 1GB RAM – $40/m
•Staging server 1: VM x2 CPU 2.0Ghz, 8GB RAM – $165/m
•Staging server 2: VM x1 CPU 2.0Ghz, 2GB RAM – $50/m
•Puppet master: VM x2 CPU 2.0Ghz, 2GB RAM – $89/m
Total: $473/m
Colo experiment
![Page 53: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/53.jpg)
![Page 54: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/54.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
![Page 55: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/55.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
![Page 56: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/56.jpg)
Colo experiment
•Dell 1U R415
•x2 8C AMD 2.8Ghz
•32GB RAM
•Dual PSU, NIC
•x4 1TB SATA hot swappable
![Page 57: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/57.jpg)
Dedicated?
![Page 58: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/58.jpg)
Dedicated?
•Hardware replacement
![Page 59: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/59.jpg)
Dedicated?
•Hardware replacement
•Managed/support
![Page 60: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/60.jpg)
Dedicated?
•Hardware replacement
•Managed/support
•Networking
![Page 61: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/61.jpg)
Colo?
![Page 62: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/62.jpg)
Colo?
•Hardware spec/value
![Page 63: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/63.jpg)
Colo?
•Hardware spec/value
•Total cost
![Page 64: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/64.jpg)
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
![Page 65: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/65.jpg)
Colo?
•Hardware spec/value
•Total cost
•Internal skills?
•More fun?!
![Page 66: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/66.jpg)
![Page 67: Puppet at the centre of everything](https://reader034.fdocuments.us/reader034/viewer/2022042614/558e7e861a28abb40b8b4578/html5/thumbnails/67.jpg)
Colo: Power
•£300-350/kWh/m
•4.5A = £520/m
•9A = £900/m