VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x...
Transcript of VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x...
![Page 1: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/1.jpg)
VMware and CPU Virtualization Technology
Jack LoSr. Director, R&D
![Page 2: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/2.jpg)
This presentation may contain VMware confidential information.
Copyright © 2005 VMware, Inc. All rights reserved. All other marks and names mentioned herein may be trademarks of their respective
companies.
![Page 3: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/3.jpg)
Emerging technologies that impact CPU virtualization
Hardware assist (VT-x/Pacifica)64-bit computingOS assist (paravirtualization)
Today’s talk:Share our perspective on emerging technologies
Overview
![Page 4: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/4.jpg)
AgendaCPU virtualization technology overview
Virtualizing the x86 architectureTrend No. 1: Hardware assist
VT-x and PacificaTrend No. 2: 64-bit computing
Benefits of 64-bit architecture64-bit guest support
Trend No. 3: OS assistVMware and paravirtualization
![Page 5: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/5.jpg)
Full Virtualization Software Stack
Hardware
VM VM VM
VMM VMMVMM VMM
ESX Server
Service Console
VMX VMXVMX VMX
Device DriversI/O Stack
SDK / VirtualCenter Agent
Third-party Solutions
DistributedServices
DRS
VMotion
DAS
Provisioning
Backup
Virt
ualC
ente
r
Managementand
DistributedVirtualization
Services
Virtual MachineMonitor
Device Drivers
VMkernel Hardware Interface
Third-Party
Agents
Storage Stack Network Stack
DistributedVirtual Machine
File System
Virtual NICand
SwitchResourceManagement
CPU SchedulingMemory SchedulingStorage BandwidthNetwork Bandwidth
EnterpriseClassVirtualizationFunctionality
VM
![Page 6: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/6.jpg)
Today’s Focus
Hardware
VM VMVM VM
VMM VMMVMM VMM
ESX Server
Service Console
VMX VMXVMX VMX
Device DriversI/O Stack
SDK / VirtualCenter Agent
Third-party Solutions
DistributedServices
DRS
VMotion
DAS
Provisioning
Backup
Virt
ualC
ente
r
Managementand
DistributedVirtualization
Services
Virtual MachineMonitor
Device Drivers
VMkernel Hardware Interface
Third-Party
Agents
Storage Stack Network Stack
DistributedVM File System
Virtual NIC &SwitchResource
ManagementCPU Scheduling
Memory SchedulingStorage BandwidthNetwork Bandwidth
EnterpriseClassVirtualizationFunctionality
![Page 7: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/7.jpg)
Virtualization SW Terminology
Virtual Machine Monitor (VMM)SW component that implements virtual machine hardware abstractionResponsible for running the guest OS
HypervisorSoftware responsible for hosting and managing virtual machinesRun directly on the hardwareFunctionality varies greatly with architecture and implementation
Hypervisor
VMM VMMVMM
Base Functionality (e.g. scheduling)
EnhancedFunctionality
![Page 8: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/8.jpg)
CPU VirtualizationThree components to classical virtualization techniquesMany virtualization technologies focus on handling privileged instructions
Memory partitioning and allocation of physical memory
Memory virtualization
Routing I/O requests between virtual devices and physical hardware
Device and I/O virtualization
De-privileging or ring compression to handle privileged instructions
Privileged instruction virtualization
![Page 9: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/9.jpg)
Handling Privileged InstructionsIn traditional systems
OS runs in privileged modeOS “owns” the hardwareApplication code has less privilege
VMM needs highest privilege levelfor isolation and performanceTraditional VMM relies on “ring compression” or “de-privileging”
Run privileged guest OS code at user-levelPrivileged instructions trap, and emulated by VMM
Ring 3
Ring 0Guest OS
Apps
Ring 3
Ring 0
Guest OS
Apps
VMM
![Page 10: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/10.jpg)
Virtualizing x86 ArchitectureDe-privileging not possible with x86!
Some privileged instructions have different semantics at user-level: “non-virtualizableinstructions”
VMware uses direct execution and binary translation (BT)
BT for handling privileged codeDirect execution of user-level code for performanceAny unmodified x86 OS can run in virtual machine
![Page 11: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/11.jpg)
Protecting the VMMNeed to protect VMM and ensure isolation
Protect virtual machines from each otherProtect VMM from virtual machines
VMware relies on segmentation hardware to protect the VMM
VMM lives at top of guest address spaceSegment limit checks catch writes to VMM area
0 4GBVMM
![Page 12: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/12.jpg)
AgendaCPU virtualization technology overview
Virtualizing the x86 architectureTrend No. 1: Hardware assistTrend No. 2: 64-bit computingTrend No. 3: OS assist
![Page 13: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/13.jpg)
Trend No. 1: Hardware AssistCPU vendors are embracing virtualization
Intel Virtualization Technology (VT-x)AMD Pacifica
These CPU technologies are a series of enhancements to aid virtualization SW
Initially focused on handling non-virtualizableinstructionsUse a trap-and-emulate modelAlternative to using binary translation
But hardware assist does not eliminate need for VMware technology
![Page 14: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/14.jpg)
VT-x/Pacifica OverviewKey feature is new CPU execution mode (root mode)
VMM executes inroot modeAllows x86 virtualizationwithout binary translationor paravirtualizationGuest state stored inVirtual Machine ControlStructures (VT-x) or Virtual Machine Control Block (Pacifica)
Root m
odeN
on-root mode
Ring 3
Ring 0
VMexit
VMenter
Guest OS
Apps
Virtual Machine Monitor (VMM)
Guest OS
Apps
![Page 15: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/15.jpg)
Limitations of Hardware Assist Initial VT-x/Pacifica hardware does not include all components of CPU virtualization solutionVT-x requires small emulator for real mode codeMemory virtualization support lacking
Not in VT-x; implementation-dependent for PacificaMemory virtualization is key to performance!
No device virtualization support
Hardware Assist
NoMemory virtualizationNoDevice and I/O virtualization
YesPrivileged instructions
![Page 16: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/16.jpg)
Future of Hardware Assist CPU vendors will add more hardware capabilities in future
Memory virtualization (Nested paging, EPT)VMware software will evolve to incorporate support for these new technologies
Adopt technologies as they enable new capabilities
Hardware Solution
Extended Page Tables/Nested Paging
Memory virtualization
Intelligent DevicesDevices and I/O
VT-x, PacificaPrivileged instructions
![Page 17: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/17.jpg)
Trend No. 2: 64-bit ComputingProgression of the x86 architecture
16-bit: 8086/8088 (1978)32-bit: 80386 (1985)64-bit: x86-64 (2003): a.k.a. AMD64, x64
x86-64 architecture brings 64-bit computing to industry-standard systems
Provides compatibility mode to run 32-bit x86 applicationsExtensions to x86 architecture
![Page 18: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/18.jpg)
64-bit Transition Has Already BegunApps exhausting limits of 32-bit address space
Consuming 1 bit of address space / yearDatabases, Java app servers, other threaded applications
Most new CPUs are 64-bit enabledAMD64, EM64T
Major OSes have been portedWindows, Linux, Solaris 10, etc.
Applications are being portedDatabases, app servers, development tools, games, etc.
![Page 19: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/19.jpg)
Virtualization And x86-64Potential questions about 64-bit transition
Do my apps run in 64-bit OS?Have drivers been ported?Are the 64-bit OSes robust?
The solution: virtualization!Great aid for 64-bit transitionEasy way to evaluate newOSesCan run 64-bit guest OSeson 32-bit host OS on 64-bit hardware)!
64-bit Hardware
VMM VMM64
32-bitVM
64-bitVM
Applications
32-bit or 64-bitHost OS
![Page 20: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/20.jpg)
Challenges of Virtualizing x86-64Initial AMD64 architecture did not include segmentation in 64-bit mode
Segmentation also missing from EM64THow do we protect the VMM?
64-bit guest support requires additional hardware assistance
Segment limit checks available in 64-bit mode on newer AMD processorsVT-x can be used to protect the VMM on EM64T
Requires trap-and-emulate approach instead of BT
![Page 21: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/21.jpg)
Flexible VMM ArchitectureFlexible “multi-mode” VMM architecture
Separate VMM per virtualmachine32-bit: BT VMM64-bit: BT or VT/PacificaVMM depending onhardware
Select mode thatachieves best workload-specific performanceSame VMM architecture for ESX Server, GSX Server, Workstation and ACE
VM VMVM VM
BTVMM32
BT/VTVMM64
BT/VTVMM64
BTVMM32
. . .
. . .
![Page 22: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/22.jpg)
64-bit Guests And WS 5.5Workstation 5.5 enables 64-bit guests
Currently in betaSimultaneously run 32-bit and 64-bit guestsRuns on 32-bit and 64-bit host OSes
64-bit Hardware
VMM VMM64
32-bitVM
64-bitVM
Applications
32-bit or 64-bitHost OS
![Page 23: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/23.jpg)
Requirements For 64-bit GuestsNewer hardware required for 64-bit guest support
AMD Opteron Rev. E or laterAMD Athlon64 Rev. D or laterIntel VT-enabled processor
How to determine that you have a 64-bit capable system?
Workstation 5.5 will automatically check to see if your CPU meets the requirementsCPU check utility also available for download on WS5.5 beta web pagehttp://www.vmware.com/products/beta/ws/
![Page 24: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/24.jpg)
Trend No. 3: OS AssistThree alternatives for handling non-virtualizableinstructions
Binary translationHardware assistOS assist or paravirtualization
AverageGoodPerformance
AverageHighVMM sophistication
ExcellentExcellentCompatibility
Para-virtualization
Hardware Assist
Binary Translation
![Page 25: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/25.jpg)
ParavirtualizationParavirtualization can address same problem as hardware assist
AverageGoodPerformance
AverageHighVMM sophistication
ExcellentExcellentCompatibility
Para-virtualization
Hardware Assist
Binary Translation
Modify the guest OS to remove non-virtualizable instructionsExport a simpler architecture to OS Cannot support unmodified OSes (e.g. Windows 2000/XP)
Poor
Paravirtualization not limited to CPU virtualizationHigher performance possible
Excellent
Relatively easy to add paravirtualization support:very difficult to add binary translation
Average
![Page 26: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/26.jpg)
Paravirtualization ChallengesXenoLinux paravirtualization approach unsuitable for enterprise use
Relies on separate kernel for native and in virtual machine Guest OS and hypervisor tightly coupledTight coupling inhibits compatibilityChanges to the guest OS are invasiveInhibits maintainability and supportabilityGuest kernel must be recompiled when hypervisor is updated
How can we deliver paravirtualization for enterprise customers?
![Page 27: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/27.jpg)
VMI Paravirtualization APIVMware proposal: Virtual machine Interface API
VMI provides maintainability & stabilityAPI supports low-level and higher-level interfacesAllows same kernel to run natively and in a paravirtualized virtual machine: “transparent paravirtualization”Allows for replacement of hypervisors without a guest recompilePreserve key virtualization functionality: page sharing, VMotion, etc.
We are gathering feedback on the API from many kernel developers and OSVs
http://www.vmware.com/vmihttp://www.vmware.com/standards/hypercalls.html
![Page 28: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/28.jpg)
VMI ParavirtualizationVMI approach to paravirtualization improves compatibilityAPI need not compromise performance compared to invasive paravirtualization
ExcellentAverageGoodPerformance
AverageAverageHighVMM sophistication
GoodExcellentExcellentCompatibility
Para-virtualization
Hardware Assist
Binary Translation
![Page 29: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/29.jpg)
VMware Paravirtualization PerformancePerformance Relative to Native
Bigger is Better
00.10.20.30.40.50.60.70.80.9
1
compile Dbench/1client [cpumicrobenchmark]
Dbench/1clientHIGHMEM [cpu
microbenchmark]
Database kernel
Xen2 Xen3-devel VMware-para
![Page 30: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/30.jpg)
VMware And Paravirtualization
Hardware
DistributedVM FileSystem
Device Drivers
StorageStack
ESX Server
ResourceManagement
VM
VMM32
VM
VMM64Para-VMM
VM
VirtualNIC &Switch
NetworkStack
VM
VMM64
VMware will support paravirtualizedLinux OSes
Another guest type when suchOS’s commercially available
Flexible architectureUse most efficienttechnique for theguest OS typeBT, VT/Pacifica, or paravirtualization
![Page 31: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/31.jpg)
Summary64-bit transition happening now
Virtualization can assist with transition64-bit guests supported in WS5.5
VMware provides flexible architecture to support emerging virtualization technologies
Multi-mode VMM utilizes binary translation, hardware assist and paravirtualizationSelect best operating mode for the workload
VMware will support paravirtualized guests as they appear in enterprise distributions
VMI offers superior maintainability/flexibilityPerforms as well as invasive paravirtualization
![Page 32: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/32.jpg)
PAC346VMware and CPU Virtualization Technology
Jack LoSr. Director, R&D
![Page 33: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/33.jpg)
Backup slides
![Page 34: VMware and CPU Virtualization Technologydownload3.vmware.com/vmworld/2005/pac346.pdf · VT-x requires small emulator for real mode code Memory virtualization support lacking Not in](https://reader034.fdocuments.us/reader034/viewer/2022052420/5a83245d7f8b9a0c748e965c/html5/thumbnails/34.jpg)
Performance of Binary TranslationBT provides many performance optimization opportunities
Fault eliminationAvoid costs of repeated virtual machine exitsBinary translator identifies faulting instructions and replaces them with special translations
Jump directly to appropriate handlers without an expensive fault
Guest and VMM share an address space:reduces context switch costs