Post on 11-May-2015
description
Virtualize with KVMTools and Enterprise usage
by Vincent Van der Kussen
#whoami
● Using Linux since 1996
● Sysadmin / Engineer / Opensource Enthousiast
● @vincentvdk
● http://blog.vanderkussen.org
So what is this KVM thing anyway?
It's NOT!
KVM : overview
Kernel-based Virtual Machine, a Kernel module that turns Linux into a hypervisor
Created by Quramnet (bought by Red Hat 2008)
Supports x86 (32 and 64 bit), s390, Powerpc
Included in Linux kernel since 2.6.20
Needs a CPU with virtualization extention !
KVM : overview
● Each vm is a process
● Each virtual cpu is a thread
● Open source
Features
CPU and memory overcommit
High performance paravirtual I/O
Hotplug (cpu, block, nic)
SMP guests
Live Migration
Power management
Features
PCI Device Assigenment and SR-IOV
KSM (Kernel Samepage Merging)
SPICE
NUMA
Not your typical VMWare workstation or VirtualBox.
VirtManager comes close but still a little rough on the edges
KVM is meant for virtualization on servers. Desktop usage is not always ideal (think vSphere, HyperV,..).
KVM : differences
Overview Tools
libvirt (api)
Virsh
Qemu
VirtManager
Libguestfs
virt-v2v
Overview tools
sVirt
virt-*
Boxgrinder
VirtIO (not really a tool)
Spice
Libvirt : The Virtualization API
Libvirt : overview
● “The Virtualization API”
● Toolkit to interact with the virtualization layer
● Written in C
● LGPL● Has bindings for common languages (Python,
Ruby, Perl, PHP, Java,..)● Supports AMQP via libvirt-qpid
Libvirt : overview
● Provides management of
– Virtual machines
– Virtual networks
– Storage
● Does NOT provide multi node management features like load balancing
● Suspend / resume support
Libvirt : terminology
Node
a physical machine
Libvirt : termonology
Hypervisor
A software layer that allows a node to run virtual
machines
Libvirt : terminologie
Domain
An instance (“virtual machine”) running on the
hypervisor
Libvirt : overview
Supports also :
XEN KVM
VMWare
LXC
OpenVZ
Microsoft HyperV
Virtualbox
virsh
Virsh overview
Command line for libvirt
Enables scripting
Libvirtd needs to be started
Most commands require root privileges
Virsh usage
virsh <command> <domain-id> [OPTIONS]virsh <command> <domain-id> [OPTIONS]
Virsh usage
Connect to hypervisor on the localhost
virsh connect qemu:///systemvirsh connect qemu:///system
Connect to hypervisor on remote system over SSH
virsh -c qemu+ssh://user@host.domain.foo/systemvirsh -c qemu+ssh://user@host.domain.foo/system
virshvirsh
or
Virsh usage
Virsh : storage
● Define storage pool● Local filesystem (ext3, ext4,...) ● Directory based● ISCSI● NFS● LVM (uses volume group)● Physical disk● multipath
Virsh : create storage pool
[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml[root@localhost ~]# virsh pool-define dir_pool.xml Pool LOCAL-STORAGE defined from dir_pool.xml
[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started[root@localhost ~]# virsh pool-start LOCAL-STORAGEPool LOCAL-STORAGE started
<pool type="dir"><name>LOCAL-STORAGE</name> <target> <path>/VMS</path> </target></pool>
Xml file ->
Virsh : create storage pool
[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no
[root@localhost ~]# virsh pool-listName State Autostart -----------------------------------------default active yes LOCAL-STORAGE active no
[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted
[root@localhost ~]# virsh pool-autostart LOCAL-STORAGEPool LOCAL-STORAGE marked as autostarted
Virsh : create VM
● VM is defined in xml file
● Manual creation of VM's disks
[root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120 [root@localhost ~]# qemu-img create /VMS/vm02-fromxml.img 5GFormatting '/VMS/vm02-fromxml.img', fmt=raw size=5368709120
● Define VM in libvirt and start it
[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml
[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started
[root@localhost ~]# virsh define vm02_from_xml.xmlDomain vm02-fromxml defined from vm02_from_xml.xml
[root@localhost ~]# virsh start vm02-fromxmlDomain vm02-fromxml started
1 <domain type='kvm'> 2 <name>vm02-fromxml</name> 3 <uuid></uuid> 4 <memory>128288</memory> 5 <currentMemory></currentMemory> 6 <vcpu>2</vcpu> 7 <os> 8 <type arch='x86_64' machine='pc-0.14'>hvm</type> 9 <boot dev='hd'/>10 </os>11 <features>12 # features such as ACPI are defined here13 </features>14 <devices>15 <emulator>/usr/bin/qemu-kvm</emulator>16 <disk type='file' device='disk'>17 <driver name='qemu' type='raw' cache='none'/>18 <source file='/VMS/vm02-fromxml.img'/>19 <target dev='vda' bus='virtio'/>20 <alias name='virtio-disk0'/>21 <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>22 </disk>23 </devices>24 </domain>
Virsh : list vm's
[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off
[root@localhost ~]#
[root@localhost ~]# virsh list --all Id Name State---------------------------------- 4 vm01 running - vm02-fromxml shut off
[root@localhost ~]#
Virsh : live migration
virsh migrate --live <guestname> qemu+ssh://<destination host>/systemvirsh migrate --live <guestname> qemu+ssh://<destination host>/system
Virsh : snapshots
● Only on qcow2 images
# virsh snapshot-create <domain># virsh snapshot-create <domain>
# virsh snapshot-list <domain># virsh snapshot-list <domain>
Name Creation Time State---------------------------------------------------
1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running
Name Creation Time State---------------------------------------------------
1295973577 2011-01-25 17:39:37 +0100 running 1295978837 2011-01-25 19:07:17 +0100 running
# virsh snapshot-restore <domain> <snapshotname># virsh snapshot-restore <domain> <snapshotname>
Qemu
qemu
● KVM istself cannot create VM's
● KVM is “just” a hypervisor
● Qemu as user space process
● Qemu communicates via /dev/kvm
qemu
● Qemu-img
– Check
– Convert
– Create
– Snapshot
– Info
Qemu
[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#
[root@HV01 ~]# qemu-img info /var/lib/libvirt/images/WIN7.img image: /var/lib/libvirt/images/WIN7.imgfile format: rawvirtual size: 12G (12884901888 bytes)disk size: 12G[root@HV01 ~]#
Inspect image
qemu
Example :
Convert raw to vmdk image
qemu-img convert -O vmdk <raw image> outimage.vmdkqemu-img convert -O vmdk <raw image> outimage.vmdk
Qemu
● file
● blkverify
● sheepdog
● blkdebug
● qcow2
● vvfat
● cow
● Raw
● vmdk
● vdi
● qcow
Supported disk formats
virtio
● "full virtualization" is a nice feature because It allows you to run any operating system virtualized.
● However, it's slow because the hypervisor has
to emulate actual physical devices.
virtio
● Virtio drivers solve this
● Virtio drivers know they run in a virtualized environment. (no emulation)
● Virtio drivers operate with hypervisor
virt-*
virt-*
● Virt-install
● Virt-viewer
● Virt-clone
● virt-top
Virt-install
[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.
[root@localhost ~]# virt-install --connect qemu:///system --graphics vnc,listen=0.0.0.0 --name=vm01 --ram 512 --disk pool=LOCAL-STORAGE,size=10,bus=virtio,cache=none /--network network=default,model=virtio /--location=http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64 Cannot open display: Run 'virt-viewer --help' to see a full list of available command line optionsDomain installation still in progress. You can reconnect to the console to complete the installation process.
● Can create VM and disks in one go
Virt-viewer
● Connect to the VM's console with – RDP – VNC – SPICE
● Default vnc port -> 5900
● virt-viewer
virt-viewer -c qemu:///system xpsp2virt-viewer -c qemu:///system xpsp2
virt-clone
● Clones an existing VM (Duh!!)
● Copies disk images
● Defines new machine based on source
– MAC address is auto generated
● Deploy many similar machines from one image
(template).
[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37
Clone 'vm01-copy' created successfully.[root@HV01 ~]#
[root@HV01 ~]# virt-clone -o vm01 -n vm01-copy -f / /VMS/vm01-copy.imgAllocating 'vm01-copy.img' | 10 GB 03:37
Clone 'vm01-copy' created successfully.[root@HV01 ~]#
virt-clone
virt-top
virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy
virt-top 20:40:19 - x86_64 2/2CPU 2666MHz 3962MB4 domains, 4 active, 4 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0CPU: 2.8% Mem: 2560 MB (2560 MB by guests)
ID S RDRQ WRRQ RXBY TXBY %CPU %MEM TIME NAME 14 R 0 0 1220 0 2.5 25.0 315:52.95 WIN7 15 R 0 4 146 42 0.2 12.0 0:54.64 vm01 11 R 0 0 0.2 12.0 15:38.34 vm02-fromxml 16 R 0 0 104 0 0.0 12.0 0:54.82 vm01-copy
Virtual machine manager
Virtual Machine Manager
● A GUI for virsh and virt-*
● Runs only on Linux/Unix
● Can discover machines running libvirtd (nodes)
● Local development / testing
Virtual Machine Manager
libguestfs
libguestfs :Converting, Inspecting and modifying VM's
libguestfs
● Manipulation of VM images
● Inspection of VM images
● Exists out of several separate tools
– List is available on http://virt-tools.org/docs/index.html
● No need to login on the VM itself
● Can be run on an active VM
What is this image?
Developers like to keep stuff
Is it still needed?
Libguestfs : virt-inspector
● What if you just have an image
● You cannot login to machine
● You would like to know what version of a
package is installed
● Generate list of all VM's
Libguestfs : virt-inspector
virt-inspector -d WIN7virt-inspector -d WIN7
[root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img [root@HV01 VMS]# virt-inspector -a /var/lib/libvirt/images/WIN7.img
Inspect the domain
Inspect an image
1 <name>windows</name>
2 <arch>i386</arch>
3 <distro>windows</distro>
4 <product_name>Windows 7 Professional</product_name>
5 <product_variant>Client</product_variant>
6 <major_version>6</major_version>
7 <minor_version>1</minor_version>
8 <windows_systemroot>/Windows</windows_systemroot>
9 <windows_current_control_set>ControlSet001</windows_current_control_set>
10 <hostname>btr-PC</hostname>
11 <format>installed</format>
1 <applications>
2 <application>
3 <name>Mozilla Firefox 6.0.2 (x86 en-US)</name>
4 <display_name>Mozilla Firefox 6.0.2 (x86 en-US)</display_name>
5 <version>6.0.2</version>
6 <install_path>C:\Program Files\Mozilla Firefox</install_path>
7 <publisher>Mozilla</publisher>
8 <url>http://www.mozilla.com/en-US/</url>
9 <description>Mozilla Firefox 6.0.2 (x86 en-US)</description>
10 </application>
11 </applications>
Libguestfs : virt-df
[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#
[root@HV01 ~]# virt-df vm02-fromxmlFilesystem 1K-blocks Used Available Use%vm02-fromxml:/dev/sdb 3516418 3516418 0 100%vm02-fromxml:/dev/sda1 495844 30573 439671 7%vm02-fromxml:/dev/VolGroup/lv_root 3652680 623180 2843952 18%[root@HV01 ~]#
● Allows you to view filesystem layout and space free/used
Virt-df <name of domain>
Libguestfs : virt-win-reg
● Modify registry of Windows machines
virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg
Libguestfs : virt-win-reg
guestfish -i <vmname>.img
<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys
guestfish -i <vmname>.img
<fs>upload viostor.sys /WINDOWS/system32/drivers/viostor.sys
virt-win-reg --merge <vmname>.img viostor-regvirt-win-reg --merge <vmname>.img viostor-reg
Manually Prepare a Windows image with virtio drivers
Libguestfs : virt-tar-out
● Create backups from files on the VM
# virt-tar-out -d vm01 /home home-vm01.tar# virt-tar-out -d vm01 /home home-vm01.tar
● Create backups from files on the disk image
# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar# virt-tar-out -a /VMS/vm01.img /home home-vm01-disk.tar
virt-v2v : migration of virtual machines
Virt-v2v : short overview
● Tool to automate migrations to KVM/RHEV
● Migrate to KVM or RHEV from
– XEN
– VMWare
– KVM
● Installs virtio drivers and reconfigures machine
● Can be scripted for bulk operations
Virt-v2v : migrate from Vmware to RHEV
Virt-v2v : migrate from VMWare to RHEV
virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name
virt-v2v -ic esx://esx.example.com/?no_verify=1 -o rhev -os storage.example.com:/exportdomain --network rhevm vm-name
Virt-v2v : migrate VMWare to KVM
sVirt : secure your host and VM's
sVirt
Virtualization introduces new risks
sVirt
“Old” days
Machines where physically separated and “attackable” via the network
sVirt
With Virtualization :
A flawed hypervisor can give access to ALL VM's running on it
Bypassing network security
sVirt
sVirt :
Apply MAC for guest and resources
MAC policy is set by hypervisor host
Based on SELinux (don't reinvent the wheel)
Auto apply or manual
[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img
[root@HV01 VMS]# ls -Z |grep vm01qemu qemu system_u:object_r:svirt_image_t:s0:c656,c817 vm01-copy.imgqemu qemu system_u:object_r:svirt_image_t:s0:c690,c972 vm01.img
Disk images
Processes
[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy
[root@HV01 VMS]# ps auxZ | grep vm01 |awk '{print $1" " $22}' system_u:system_r:svirt_t:s0:c690,c972 vm01system_u:system_r:svirt_t:s0:c656,c817 vm01-copy
A lot of tools but what to do with them?
RHEV is first example
Build your own stack?
Development environment?
Enterprise usage
RHEV 2.x
● RHEV (Red Hat enterprise Virtualization)
– Red Hat started adopting KVM (actually they bought it)
– Released 2.1
– Running on Windows platform ( the horror.. )
– RHEV 2.2 will be latest release in 2.x series
RHEV 3.0
● Port from .Net to Java
● Manager runs on Jboss
– no more windows server side
● Hypervisor based on RHEL 6
– Gives performance increase
● Local storage
● Reporing engine included
● WAN optimized SPICE
RHEV 3.0
● Power User portal
– Users can manage their own environment
● RESTful API
● Will be open sourced
RHEV adoption
OVA
● Open Virtualization Alliance● Foster the adoption of KVM as an enterprise-
ready open virtualization solution ● Accelerate the emergence of an ecosystem of
third-party solutions around KVM.● Encourage interoperability, promote best
practices, and highlight examples of customer successes.
Ova : members
specvirt
● Virtualization benchmark (score@ 'n' VM's)
● KVM
– 3894@240
● VmWare ESX 4.1
– 3723@228
conclusions
● KVM is till young
● Adoption is growing
● Builds on Linux features (feature velocity)
– Selinux, Cgroups, Scheduler, ...
● A lot of community activity
● scalability
● Growing ECO system
● No vendor lockin
● OVA (open virtualization alliance)
Questions?
http://www.flickr.com/photos/kalavinka/4617897952/
http://www.flickr.com/photos/zakh/337938459/
http://www.cyberciti.biz/faq/linux-kvm-vnc-for-guest-machine/
http://www.centos.org/docs/5/html/5.2/Virtualization/chap-Virtualization-Managing_guests_with_virsh.html
http://heprc.phys.uvic.ca/sites/heprc.phys.uvic.ca/files/reports/vliet-wtr.pdf
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Virtualization/index.html
Grind out appliances
BoxGrinder : overview
● Create a VM “offline”
● Runs on JBOSS
● Use puppet/chef/.. to configure it further
● Move from Fedora to RHEL by adjusting the file
●