Virtual Infrastructure:kvm, uvb + libvirt

Bryan McLellanhttp://loftninjas.org btm@loftninjas.org

Senior Systems AdministratorWidemile

Widemile is awesome

I work there, and thus the world is a better place for all of humanity.

What I got for “free”

•Identical Blades

•Running non identical Debian installs (etch/sid, i386/amd64)

•Running different versions of vmware-server

• OS, software, installed by hand

• Configure using notes in an outlook public folder

•Host database tracked with DNS

vm04-something.test.widemile.com

Problems

• Lack of Homogeneity

• Monkey configuration sucks

• Documentation offensive

http://www.flickr.com/photos/annebuzz/2666654754/

What I did

• Lack of Homogeneity– debian pxe install with preseeds

• Monkey configuration sucks– puppify (configuration management)– capistrano (ruby scripting over ssh)– iclassify (node classification)

• Documentation offensive– Configuration management is self documenting– Wikify the notes

http://www.flickr.com/photos/jpasden/67513019/

We can always do better

What upset me (nobody else cares)

• PXE installs take soooooo long• Backporting too many packages to debian• Interacting with VMware programmatically

too much work.• VMware UIs cost $$• Dreams: cpu/ram hotplug, live migration

How I Learned to Stop Worrying and Love the Bomb

• PXE installs take soooooo long– deb-bootstrap with ubuntu-vm-builder (2.5min)

• Backporting too many packages to debian– switch to Ubuntu!

• Interacting with VMware programmatically too much work.– libvirt is much easier and cleaner

• VMware UIs cost $$– Libvirt is free

• Dreams: cpu/ram hotplug, live migration– Word on the street is that kvm + libvirt support these– I don’t think they’re there yet though.

libvirt – virtualization API

Started at Redhat ‘s Emerging Technology GroupWas Xen focused, but Redhat recently bought Qumranet, makers of KVMUbuntu/Canonical picked KVM as virtualization platform of choicelibvirt doesn’t really support all thatProvides CLI management (virsh)Provides GUI management (virt-manager)

virsh

virt-manager

ubuntu-vm-builderBash script wrapper for deb-bootstrap + qemuapt-get install ubuntu-vm-builder

capistrano script:vmware: 102 linesuvb: 12 lines

ubuntu-vm-builder kvm hardy --addpkg openssh-server -d /srv/kvm/#{hostname} --domain #{domain} --hostname #{hostname} --mem #{memory} --mirror http://ubuntu.widemile.com/ubuntu--libvirt qemu:///system--bridge #{eth0}

ubuntu-vm-builderBash script wrapper for deb-bootstrap + qemuapt-get install ubuntu-vm-builder

capistrano script:vmware: 102 linesuvb: 12 lines

ubuntu-vm-builder kvm hardy --addpkg openssh-server -d /srv/kvm/#{hostname} --domain #{domain} --hostname #{hostname} --mem #{memory} --mirror http://ubuntu.widemile.com/ubuntu--libvirt qemu:///system--bridge #{eth0}

Not a lot of enterprise features yetBridging vs NAT

Root password / creates user

kvm• The cool hypervisor on the block (ubuntu, redhat)• Kernel modifications are mainlined (unlike xen)• Super fast / lightweight (feel the wind in your hair)• Growing fast

class kvm { case $lsbdistid { "Ubuntu": { # Only Ubuntu releases are new enough for this

package { "kvm": ensure => present; "libvirt-bin": ensure => present; "ubuntu-vm-builder": ensure => latest; }

file { "/srv/kvm": ensure => directory, group => admins, mode => 0775, }

# Permissions required for remote access # Puppet doesn't support managing the libvirtd group members through

the group type file { "/var/run/libvirt/libvirt-sock": group => admins, require => Service["libvirt-bin"]; "/var/run/libvirt/libvirt-sock-ro": group => admins, require => Service["libvirt-bin"]; }

service { "libvirt-bin": ensure => running, hasstatus => true, require => Package["libvirt-bin"], }

# required by u-v-b exec { "devmapper-autoload": command => "/bin/echo dm_mod >> /etc/modules", onlyif => "/usr/bin/test `grep -c '^dm_mod' /etc/modules` -lt 1", } exec { "devmapper-load": command => "/sbin/modprobe dm_mod", onlyif => "/usr/bin/test `/bin/lsmod | grep -c '^dm_mod'` -lt 1", } realize Group["libvirtd"] } }}

http://loftninjas.org