An Introduction to Virtualization on Linux Highlighting the ...

Introduction Types Innovation Xen

An Introduction to Virtualization on LinuxHighlighting the Open Source Xen Hypervisor

Austin [email protected]

http://uberhip.com/virtualization

February 8, 20081 / 65

http://www.jumpbox.com

Introduction Types Innovation Xen Definition Motivation Benefits Pitfalls

Introduction

2 / 65


(Platform) Virtualization Definition

... an abstraction layer that allows multiple virtual[computers], with heterogeneous operating systemsto run in isolation, side-by-side on the same physicalmachine.- VMWare

Notice: We are assuming Consumer x86 Hardware

3 / 65

http://info.vmware.com/content/GLP_BizVirt


What Is (Platform) Virtualization

Multiple Operating Systems runningsimultaneously

Fully isolated machines with virtual hardware,possibly including BIOS and different CPU

VMs appear to Host as files/processes(approximately)

One host can have many guests

4 / 65


What Does It Look Like

The disks and othercomponents are generally fileson a Host OS.

GUI Management Console

CLI Management Tools

5 / 65


Motivations For Virtualization

Server Consolidation power savings, increased hardwareutilization

Service Isolation simplfying administration of a given service

Disaster Recovery inexpensive live standby, full systembackups, incremental system backups

Development Testing on Multiple OS Versions (Service Packs,Language Versions), Cheap Development Copies

OS Independence Run the OS you want while providing accessto OS/applications

6 / 65








7 / 65








8 / 65








9 / 65








10 / 65


Benefits Of Virtualization

Suspend/resume snapshot disk and memory state

Multiple Snapshots store an arbitrary number ofbranching snapshots

Portability OS is decoupled from the hardware,cloning, virtual appliances

Live Migration Extension of portability, but you cando it while the OS is running

11 / 65







12 / 65







13 / 65







14 / 65


VMWare Snapshot Manager

Multiple snapshots, maintain the state of yourentire virtual machine at a given point in time.

15 / 65


Pitfalls of Virtualization

These roses do have thorns.

16 / 65


Increased Complexity

Networking - More tubes!

17 / 65


Increased Complexity

Added layers of complexityGuest OSVirtualization PlatformVirtualization ToolsSAN

Suspend/Resume IssuesLaggy ClocksCollisions - IPs, MACsRelocation - Network Settings

18 / 65


New and Improved Fat Finger Catastrophes

single points of failuresudo rm -rf datacenter

sudo xm destroy 100sudo vzctl destroy 100

19 / 65


New and Improved Fat Finger Catastrophes

single points of failuresudo rm -rf datacenter

sudo xm destroy 100sudo vzctl destroy 100

20 / 65


Now with Added Security Issues

New Hardware To Exploit - BluePill

New Software Layer to Exploit

New Software Layer to keep updated

Shared Hardware Information Leakage(CPU/Network)

21 / 65

Introduction Types Innovation Xen Emulation Full Virtualization Paravirtualization Partitioning

Types

22 / 65


Types of Virtualization

Goal of Virtualization SystemDesktop VirtualizationServer Virtualization

Virtualization MechanismHardware EmulationOS/API EmulationFull VirtualizationHardware Assisted Full VirtualizationParavirtualizationOS Partitioning

23 / 65


Hardware Emulation

Supports Different CPU Architectures

Emulation at the CPU Instruction Level

Tends to be slow

Example Implementations: QEMU, Bochs,MAME, basilisk2, PearPC

24 / 65


Hardware Emulation

25 / 65


OS/API Emulation

Allows execution of a guest application on thehost OS without running full guest OS.

Reimplementation of the guests system andlibrary interfaces on the Host platform

Faster than emulation without the overhead ofrunning the entire guest OS.

Example Implementations: Wine, LxRun

26 / 65


OS/API Emulation

27 / 65


Full/Native Virtualization

Guests must have same CPU Architecture ashost.

Host emulates hardware devices that Guest mustsupport.

Gets tricky without hardware support, because itmust dynamically scan executing code fortrouble (See VirtualBox site for description)

Examples: VMWare Player, GSX, and ESX,Parallels Workstation, VirtualBox

28 / 65


Full Virtualization

29 / 65


Hardware Assisted Full Virtualization

Guests must have same CPU Architecture ashost.

Runtime scanning of Guest code unnecessary.

Guests are unmodified.

Hardware support must be present: Intel VT orAMD SVM extensions

Example Implementations: Xen HVM,VirtualBox, KVM

30 / 65


Hardware Assisted Full Virtualization

31 / 65


Paravirtualization

Requires Guest OS Modification which avoidsthe runtime scanning of guest code.

Slightly faster than Full Virtualization.

Guest Applications do not need modification.

Example Implementations: Xen, Lguest

32 / 65


Paravirtualization

33 / 65


Xen Paravirtualization

34 / 65


Partitioning/OS Level Virtualization

Requires host kernel modifications.

Guests share kernel.

Containers or Jails.

Example Implementations: OpenVZ,Linux-VServer, FreeVPS

35 / 65


Partitioning/OS Level Virtualization

36 / 65

Introduction Types Innovation Xen Appliances Hosting Management Development

Innovation

37 / 65


Virtual Appliances - Desktop

“A virtual appliance is a minimalist virtual machineimage designed to run under virtualizationtechnology.” – “Virtual Appliance”, WikiPedia

Anonymity/Privacy

Restricted Environments

Corporate Remote Access

Application Access

Development Environment

Check out Moka5 or VMWare

38 / 65

http://www.moka5.com/

http://vmware.com/appliances/


Virtual Appliances - Server

Turn key server applications that encapsulateSystem Administrator skill.

Deployment platform option

Reproducible Infrastructure Components

Private Application Environment

Reduce installation support costs for ISVs

Check out JumpBox, rPath, and VirtualAppliances.net

39 / 65


http://www.rpath.com

http://virtualappliances.net/

http://virtualappliances.net/


Virtualization In Hosting

Old SchoolMore Isolation than Share hostingCheaper than dedicated hardwareUML, Chroot Jail

New SchoolOpenVZ, Virtuosso, XenGeneric Xen VPS - VPSLinkBoutique Xen VPS - RailsMachine, EngineYardGeneric Utility Computing - Amazon’s EC2Enterprise Utility - 3tera

40 / 65

http://vpslink.com/

http://railsmachine.com/

http://www.amazon.com/b?ie=UTF8&node=201590011

http://www.3tera.com/


Management Tools

Enomolism - Xen/EC2 WebUI

ConVirt/XenMan - GTK Xen(KVM) UI

Ganeti - Google contributed CLI Xen ClusterManager

Xen-tools - CLI Xen Tools, provisioning

virt-manager - RedHat’s Desktop UI

Cobbler - RedHat Provisioning System

Explosion of third party tools for the commercialplatforms.

41 / 65

http://www.enomalism.com/

http://xenman.sourceforge.net/

http://code.google.com/p/ganeti/

http://www.xen-tools.org/software/xen-tools/

http://virt-manager.et.redhat.com/

http://cobbler.et.redhat.com/index.php


Development Tools

libvirtPlatform APIs

VMWare Vix (Ruby and Python)VMWare Infrastructure SDKXen XML-RPC API

Open Source Guest Tools

Wrapping Command Line Utils

42 / 65

http://rubyvmware.rubyforge.org/

http://sourceforge.net/projects/pyvix

http://www.vmware.com/support/developer/vc-sdk/

http://wiki.xensource.com/xenwiki/Xend/XML-RPC

Introduction Types Innovation Xen Installation VM Creation Usage

Xen

43 / 65


Xen OpenSource

Started as Academic research project atCambridge, this makes it somewhat unique

Was released as Open Source Project

Controlling company acquired by Citrix

Xen.org remains Independant, Open Source andActive

44 / 65

http://xen.org


Xen References

Dell’s Overview of Xen [mirror]

Xen 3.0 Users Manual [mirror]

Xen 3.0 Interface Manual [mirror]

Xen and the Art of Virtualization [mirror]

Xen and the Art of Repeated Research [mirror]

Live Migration of Virtual Machines [mirror]

45 / 65

http://www.dell.com/downloads/global/power/ps3q05-20050191-Abels.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/Xen-Overview-Abels.pdf

http://www.cl.cam.ac.uk/research/srg/netos/xen/readmes/user.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/Xen-User-Manual.pdf

http://www.cl.cam.ac.uk/research/srg/netos/xen/readmes/interface.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/Xen-interface.pdf

http://www.cl.cam.ac.uk/netos/papers/2003-xensosp.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/Xen-sosp-2003.pdf

http://www.clarkson.edu/class/cs644/xen/files/repeatedxen-usenix04.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/xen-repeated.pdf

http://www.cl.cam.ac.uk/netos/papers/2005-migration-nsdi-pre.pdf

http://uberhip.com/godber/plug/Xen_Primer/assets/xen-migration.pdf


Xen General Info

HVM Modeegrep "vmx|svm" /proc/cpuinfo

Windows

More guest isolation

32bit Dom0 64bit Dom0

32bit PV DomU Yes Kernel must be 64bit64bit PV DomU No Yes32bit HVM DomU Yes Yes64bit HVM DomU No Yes

46 / 65


Xen Paravirtualization

47 / 65


Xen Installation On RedHat and Fedora

Nicely integrated with recent Fedora RedHatEnterprise releases

Xen can be installed at initiall installation timeor after the factyum groupinstall ’Virtualization’

LibVirt abstraction away from the underlyingVirtualization platform

Good Online resources:http://fedoraproject.org/wiki/Tools/Xen

48 / 65


RedHat Virtual Machine Manager

RedHat’s libvirt based GUI Management tool forVM Creation, monitoring, and configuration.

49 / 65


Xen Installation On Ubuntu 7.10

Xen and most of the associated tools are availablefrom the Ubuntu Universe repository, so installationis as simple as:

sudo apt-get updatesudo apt-get install ubuntu-xen-serversudo apt-get remove --purge network-managersudo vim /etc/xen/xend-config.sxp# Now comment out the dummy and enable network bridge#(network-script network-dummy)(network-script network-bridge)sudo rebootsudo xm list

50 / 65


Xen On Ubuntu Details

There are a few new packages installed

linux-image-2.6.22-14-xen Dom0’s Xen Patched Kernel

xen-hypervisor-3.1 contains /boot/xen-3.1.gz

xen-ioemu-3.1 HVM IO Emulation - QEMU

xen-utils-3.1 Generic Xen Utilities

libc6-xen Xen Safe libc6

51 / 65


Preparation Of A Virtual Machine

You can do it yourself, use a tool, or find a premadeone. The general process for makine one manuallyis as follows:

1 Choose Your Virtual Disk Type

2 Create That Disk Type

3 Fill ’er up

4 Tweak The Image

5 Write the Config File

52 / 65


Chosing A Virtual Disk

Physical Block DevicesTraditional Block DeviceLVM Block Device

Fancy Block DevicesiSCSIAoEDRBD, NBD

Loopback Block DevicesOver NFS? Possible under specific circumstances.How about just using a Xen VM with an NFS Root?

53 / 65

http://www.debian-administration.org/articles/505


Creating A Virtual Disk

Create a sparse root disk and a non-sparse swapdisk.

sudo mkdir -p -m 0777 /vms/demo/dd if=/dev/zero of=/vms/demo/root.img bs=1 count=0 seek=5Gdd if=/dev/zero of=/vms/demo/swap.img bs=1G count=1

mkfs.ext3 -F /vms/demo/root.imgmkswap /vms/demo/swap.img

54 / 65


Filling The Virtual Disk

Debootstrap and RPMstrap are your friends here:

sudo mkdir /mnt/gutsysudo mount /vms/demo/root.img /mnt/gutsy -o loopsudo apt-get install debootstrapsudo debootstrap gutsy /mnt/gutsy# The RedHat equivalent# yum --installroot=/mnt/gutsy -y groupinstall Base

55 / 65


Tweak Some Things

While the disk is still loopback mounted, it is agood time to make a few changes:

cp -r /lib/modules/‘uname -r‘ /mnt/gutsy/lib/modules/cat > /mnt/gutsy/etc/fstab <<EOF/dev/sda1 / ext3 rw,errors=remount-ro 0 1/dev/sda2 none swap defaults 0 0none /proc proc defaults 0 0EOFecho "gutsy" > /mnt/gutsy/etc/hostnameecho "127.0.0.1 localhost" > /mnt/gutsy/etc/hostsecho "127.0.1.1 gutsy" >> /mnt/gutsy/etc/hosts

56 / 65


Tweak Some More Things

cat > /mnt/gutsy/etc/network/interfaces <<EOFauto loiface lo inet loopback

auto eth0iface eth0 inet dhcpEOFecho \

"deb http://archive.ubuntu.com/ubuntu gutsy main" \> /mnt/gutsy/etc/apt/sources.list

mv /mnt/gutsy/lib/tls /mnt/gutsy/lib/tls.disabled

57 / 65


Tweak a Few Last Things and Unmount

# These are Ubuntu 7.10 Dom0 bug workarounds...chroot /mnt/gutsy update-rc.d -f hwclock.sh removechroot /mnt/gutsy update-rc.d -f hwclockfirst.sh removechroot /mnt/gutsy rm /etc/udev/rules.d/85-hwclock.rules

umount /mnt/gutsyrmdir /mnt/gutsy

58 / 65


The Xen DomU Config File

Create a config file: /etc/xen/gutsy.cfg

kernel = "/boot/vmlinuz-2.6.22-14-xen"ramdisk = "/boot/initrd.img-2.6.22-14-xen"builder=’linux’memory = 256name = "gutsy"vcpus = 1vif = [ ’bridge=xenbr0’ ]disk = [ ’file:/vms/demo/gutsy/root.img,sda1,w’,

’file:/vms/demo/gutsy/swap.img,sda2,w’ ]root = "/dev/sda1 ro"extra=’xencons=tty1’ #Ubuntu 7.10 bug workaround

59 / 65


The Alternative - Xen-Tools

Xen-Tools can do it all for you, just edit/etc/xen-tools/xen-tools.conf and run:

sudo xen-create-image --size=2Gb --swap=128Mb \--dir=/vms/ --hostname=testxen \--ip=10.6.6.233 --ide --force

If you want to roll your own based on what I showedhere, I have a starter script onhttp://uberhip.com/virtualization

60 / 65


Running the DomU

To start the VM we have just created:

xm create /etc/xen/gutsy.conf# or you could have done to# start connected to the console# xm create -c /etc/xen/gutsy.confxm listxm console gutsy# to exit use CTRL+]

Now you can login as root, there IS NO ROOTPASSWORD.

61 / 65


DomU Actions

Common Commands:

create, destroy, reboot, shutdown, rename

Informational Commands:

top, info, list, uptime, dmesg, log

Checkpoint, resume, migrate:

xm save gutsy gutsy.chkxm restore gutsy.chkxm migrate --live gutsy host2.ournetwork.com

62 / 65


Manipulating Block Devices

Attach a new formatted loopback file from Dom0:

xm block-list gutsyxm block-attach gutsy file:/vms/demo/test.img /dev/xvdb wxm block-list gutsy # new Vdev is used for block-detach

Now, inside the gutsy DomU:

dmesg | tail # => xen-vbd: registered block device ...mount -t ext3 /dev/xvdb /mnt/mount # shows new mounted block deviceumount /mnt

Detaching Block Device

xm block-detach gutsy 51728 # Vdev from above

63 / 65


Colophon

Presentation produced using the Beamer class inLATEX using the Kile editor on Ubuntu 7.10.

Illustrations were created with Inkscape andOmnigraffle.

64 / 65

http://latex-beamer.sourceforge.net/

http://www.latex-project.org/

http://kile.sourceforge.net/

http://www.ubuntu.com/

http://www.inkscape.org/

http://www.omnigroup.com/applications/omnigraffle/


Thank You

Austin [email protected]

http://uberhip.com/virtualization

65 / 65


An Introduction to Virtualization on Linux Highlighting the ...

Documents

Transcript of An Introduction to Virtualization on Linux Highlighting the ...