The Grid and Virtualization

Orran KriegerSr. Technical Staff Member

Vmware

2

• A level of indirection between hardware and software

• Virtual Machines: a full abstraction of a physical machine• Safely and efficiently multiplex virtual hardware on physical hardware

• Benefits: resource sharing, portability, compatibility

Physical machine

Virtual machine

What is Virtualization?

3

Before Server Virtualization: Single OS image per machine Software and hardware tightly coupled Running multiple applications on same

machine often creates conflict Underutilized resources Fault/security vulnerability

Server Virtualization

4

Before Server Virtualization: Single OS image per machine Software and hardware tightly coupled Running multiple applications on same

machine often creates conflict Underutilized resources Fault/security vulnerability

After Server Virtualization: Virtual machines (VMs) break 1-to-1

dependency between OS and HW Manage OS and application as single

unit by encapsulating them into VMs Hardware-independent: they can be

provisioned anywhere Strong isolation between VMs

Server Virtualization

5

VMMVMM VMMVMM

VM Encapsulation and Mobility

Suspend

Resume

Snapshot

Clone

Migrate

Record

Replay

etc.

Virtual Machines are more versatile than Physical MachinesVirtual Machines are more versatile than Physical Machines

LogLog

6

VMkernelVMkernel

HAL and Device DriversHAL and Device Drivers

Resource ManagementResource Management

NetworkingNetworkingStorageStorage

RHEL3-basedRHEL3-basedService ConsoleService Console

HelpersHelpers VMMVMM

AgentAgent AgentAgent……

VMMVMM

Disk Footprint:Disk Footprint: 2 GB2 GB32 MB32 MB

RPMRPM

VMMVMM

Disk Footprint:Disk Footprint:

Traditional ESX Server

Percent of PatchesPercent of Patches >50%>50%

98%98% 2%2%

7

ESX Server 3i: Thin Virtualization!

VMkernelVMkernel

Resource ManagementResource Management

NetworkingNetworkingStorageStorage

RHEL3-basedRHEL3-basedService ConsoleService Console

VMMVMM

AgentAgent AgentAgent……

VMMVMM

RPMRPM

VMMVMMHelpersHelpers

HAL and Device DriversHAL and Device Drivers

Disk Footprint:Disk Footprint: 2 GB2 GB32 MB32 MBDisk Footprint:Disk Footprint:

Percent of PatchesPercent of Patches >50%>50%

98%98% 2%2%

Small footprint with OS-independenceVirtualization should have the reliability, security, and performance of hardware

Less code and fewer interfaces:Compact 32MB footprint

CIM- and remote CLI-based management

Aids both security and reliability (no legacy)

Integrated in server systems:Hardware is certified and ready-to-run

Simple start up experience

Launched with IBM, HP, Dell, FSC, NEC.

In the next few years, virtualization will be pervasive embedded in all HW

8

Grid and grid-like technologies are the future

• Web applications being developed with scale-out frameworks

• MapReduce and related frameworks…, Data Intensive Supercomputing

• Utilities inside companies • Hosting companies like Rackspace• Shared utilities like Amazon’s EC2• The rise of SAAS like salesforce.com • HPC capacity and capability systems

Are both technologies really going to be pervasive?Are both technologies really going to be pervasive?Grid and Virtualization are complementary:Grid and Virtualization are complementary:•Grid: motivated by analogy of power gridGrid: motivated by analogy of power grid•Virtualization: converts computation into a fungible Virtualization: converts computation into a fungible commodity; hugely simplifies realization of grid commodity; hugely simplifies realization of grid

9

Traditional Grid Adoption Challenges

• Security• Grid jobs can potentially compromise security on host machine

• Intruders may piggy-back on Grid jobs to attack the system

• Isolation• Grid jobs may “eat” into host machine's resources

• Buggy grid job may crash host system

• OSes often can’t control resource usage between Grid jobs• Forces single job per host models

• OS heterogeneity limits number of Grid nodes available for a job

• Service level guarantees• Enterprise grid jobs need guarantees on completion time

• In many cases, applications must be re-written for Grid

10

Traditional Grid Adoption Challenges (Contd)

• Provisioning• Increased activity on host system can slow down Grid jobs

• Utilization• Dedicated machines for Grid jobs lead to severe under-utilization

• Legacy applications fail to utilize multi-core features

• Management• Grids lead to increase in number of “nodes”

• Management of these nodes is a big challenge

• Infrastructure for application virtualization• Grid middleware lack support from infrastructure layer for on-demand

deployment of application instances

• Grid implementation limited by having to use commodity OSes• Large attack surface, high overhead, …

11

Virtualization in Grid Computing• “Any problem in computer science can be solved

with another level of indirection” (David Wheeler)

• Virtualization provides a natural level of indirection to address Grid Computing challenges• Inherent “cures” to Grid adoption issues

• Same solution applies to broader utility, hosting, enterprise, web … marketplace

12

Benefits of Virtualization in Grids

• Virtual machines (VM) provide robust security

• A compromised VM can be isolated without affecting the host

• VMs provide excellent isolation

• A Virtual Grid node (VGN) crash has no effect on other VGNs or host

• VGNs can guarantee service level times

• VMs can be provisioned with required explicit resource reservation• CPU bandwidth, memory, disk space, disk bandwidth, etc

• Ensures the grid job has the muscle power to complete within the SLA

• Virtualization reduces data centre complexity

• Abstracts out infrastructure heterogeneity

13

Benefits of Virtualization in Grids (Contd)

• No resource usage creep with VMs• Provisioning places limit on resources that can be consumed

• Very useful for “resource scavenging” Grids• Host performance does not deteriorate

• Multiple VGNs on a single physical machine• Increases utilization of Grid boxes

• Enhances workload management capability of the Grid• Additional VGNs can be started and deployed instantly

• On-demand provisioning of VGNs eliminates OS heterogeneity issues

• As many VGNs of a particular OS as needed

14

Benefits of Virtualization in Grids (Contd)• Dynamic resource management

• (1) Vary resource allocated to a VGN based on load/requirement

• (2) Move VGN to a more/less powerful host

• Provisioning additional grid nodes becomes trivial• Maintain repository of pre-created VM images

• Choose and bring-up VGN based on requirement

• Ease of management• Infrastructure can be managed through virtual machine management

solutions

• Maintenance and upgrade of VMs easier than physical servers

• Custom OS support• Special purpose OSes, specialized for Grid applications can be

developed, resulting in major performance, reliability, simplicity advantages.

• Lets the Grid application get much closer to the metal.

15

Grid Computing

GridGrid

MiddlewareMiddleware

GridGrid

MiddlewareMiddleware

Grid JobGrid Job

Grid JobGrid Job

Grid JobGrid Job

Grid JobGrid Job

Grid JobGrid JobGrid JobGrid Job

Jobs IN/OUTJobs IN/OUT

16

Virtualization based Grid

Grid jobGrid job Other JobOther Job

Grid MiddlewareGrid MiddlewareGrid MiddlewareGrid MiddlewareJobs IN/OUTJobs IN/OUT

•Grid middleware can focus on “core” Grid issues.

• Virtual infrastructure provides simplified abstraction used by Grid middleware and other applications.

17

VMware’s GRID enablement features

• Base features: low-overhead, strong isolation…• Integrated management of virtual and physical

infrastructure. • Guaranteed quality of service:

• VMotion to move tasks between physical machines• Distributed resource scheduling

• Reliability:• Snapshotting for checkpointing of long running jobs • High availability

• Cheap creating of VMs• memory overcommit• templates & clones

• Special purpose appliances

18

Towards Distributed Virtualization

RESOURCE RESOURCE POOLPOOL

RESOURCE RESOURCE POOLPOOL

RP 1BRP 1B

Machine VirtualizationMachine VirtualizationPhysical MachinesPhysical Machines Distributed VirtualizationDistributed Virtualization

RP1ARP1A

OSOS OSOS

19

VMotion™

• Migrate running VMs from one server to another• Encapsulation and HW independence are the key enablers

• To minimize service disruption:• Iteratively pre-copy VM memory from source to destination• Store virtual disks on networked storage: FC, iSCSI, NAS

• VGNs moved across ESX hosts with zero downtime; No need to restart Grid jobs• Allows efficient usage of hardware resources and improves overall hardware

utilization• Similar VGNs with can be grouped together for better performance

20

Storage VMotion

• Storage independent migration of virtual machine disks

• Zero downtime to virtual machines

• LUN independent

• Supported for Fibre channel SANs

21

Automatic load balancing across hosts

Distributed ResourceDistributed Resource

Scheduling (DRS)Scheduling (DRS)Dynamic BalancingDynamic Balancing

Continuous OptimizationContinuous Optimization

22

Adding and removing hosts

Hot-plug machinesHot-plug machinesAdd/remove capacity on demandAdd/remove capacity on demandDistributed power optimizationDistributed power optimizationImprove application availabilityImprove application availability

DRSDRS

XX

23

HWHW HWHW HWHW HWHW HWHW

HWHW HWHW HWHW HWHW HWHW

HWHW HWHW

HWHW HWHWHWHW

HWHW

HWHWHWHW

Transforming capacity management

VIRTUALIZEDVIRTUALIZEDPHYSICALPHYSICAL POOLEDPOOLED

RRPP11

RRPP22

Logical Resource Pooling (RP)Logical Resource Pooling (RP)Distributed Resource Scheduler (DRS)Distributed Resource Scheduler (DRS)

24

Firewall SW

Linux

FIREWALLVirtual Appliance

Apache

Linux

Linux

LinuxTomcat

mySQLCRMVirtual Appliance

What is a Virtual Appliance

• Pre-built, pre-configured and ready-to-run software application packaged with the OS inside a Virtual Machine.

• Or packaged inside multiple Virtual Machines

25

•Appliance OSes can be simplified or customized

• Don’t need complex hardware management

• Don’t need broad application support

• Highly-customizable, application-specific OS

• Include only what application needs.

• Look at hardware appliance operating systems for examples

Hardware Mgmt.

Application Support

ApplicationApplication

Operating System

Virtualization Layer

Virtual Appliance OS

26

• Virtualization and Grid will both be pervasive technologies

• Virtualization will increasingly be embedded on all server platforms.• Various forms of utility computing are increasingly dominating many markets.

• Virtualization and Grid are complementary.• Its time to start working together:

• What is the right interface between Grid middleware and virtual infrastructure?• What are the requirements of the Grid? Determinism for collective operations? Fast

start up of VM? Fast Clone…

• Its time to write Grid appliances/OSes. • How tiny can they be? Plan9?• Does a Library OS model make sense? Libra/Prose? • Is there any paravirtualization support that would be useful?

Orran Krieger <okrieger@vmware.com>

Concluding remarks

27

Thanks

• Hariprasad Nellitheertha

• Chinmay Albal

• Radhika Kulkarni

• Larry Rudolph; Steve Strassman; Scott Davis; Jack Lo

28

29

VMware Memory Overcommit

• Unique VMware Feature

• Host system memory is usually the resource bottleneck

• VMware memory management technology allows an average of 2:1 memory overcommit (and oftentimes greater)

Transparent Page SharingTransparent Page Sharing Memory ballooningMemory ballooning

30

HW Independence & Transparency

Run a virtual machine on any PC without modification. Break

dependence between OS & HW

Partitioning

Run multiple virtual machines simultaneously on a single PC

Isolation

The virtual machine is isolated from Host OS, other VMs on the

same PC

Encapsulation

Entire virtual machine is saved in files. Can be moved and copied

like any other file.

Key Properties of Virtualization