2
Setting the Stage
This presentation will discuss the usage of Linux as a base component of hypervisor
components particularily in the context of creating a base for a cloud hosting
framework.
The basis for the discussion is formed by combining technologies like KVM/QEMU,
OpenVSwitch, OpenFlow and OpenStack to form a technical base for the solution, and
digging into particular problem areas to see where the experience from the embedded
world can help in creating a more competent experience.
The presentation also discusses the general characteristics of the above technologies
to create an overview of the problem area. In particular the issues that are key to
creating a competetive solution are the responsiveness and throughput of the system,
the capability for providing logging and tracing facilities across the hypervisor and
guest domains, the seamless integration of the components to provide stability and
maintainability and providing a suitable level of security throughout the layers of SW
in the system.
Agenda
Key Components and Architecture
Main Architecture
OpenStack
OpenvSwitch
QEMU/KVM
Virtualization Problems and Solutions
Further Work on the Topic
Example Hypervisor Solution Architecture
Hardware
Carrier Grade
Services
(HA, Tracing,
Logging,
Security,
CGL)
OS layer
and base services
KVM/QEMU incl. HW
accelerationLXC
Management
and
Provisioning
SW and
Services
OpenStack
OpenFlow
Guest VMGuest
Container
OpenVSwitch
QEMU and KVM
-based
hypervisor solutions
At least it will be looking forward
Features
QEMU: Comprehensive architecture support
X86, ARM, MIPS, PPC, etc
QEMU: Support for particular board configurations
QEMU :Live Migration of guests
KVM: Integration with HW virtualization capabilities for x86,
PPC, ARM
Best support on x86 currently
ARM seeing a lot of work currently
OpenVSwitch
OpenVSwitch features
Virtual switching ports and connection to physical switch HW
Support for OpenFlow control
Support for standards such NetFlow, sFlow, SPAN
What does it provide
Facilitates live-migration
Much easier to replicate state between hosts
Abstraction API
Similar interfaces to all switches accessed
Performance
OpenFlow
Software Defined Networking (SDN) means to separate
the control plane from the data plane in network
switches and routers to different components
The expectation is that this leads to massively more
scalable network infrastructure
The initiative for the standardization of OpenFlow is
being led by major end-users like Google and Facebook
and not really the equipment vendors
OpenFlow is split into target and controller domains
API is in essence a adaptation layer
Both types of OSS SW for Linux available
Open Stack Cloud Management
OpenStack is a cloud management system that controls large pools of
compute, storage, and networking resources throughout a datacenter, all
managed through a dashboard that gives administrators control while
empowering their users to provision resources through a web interface.
OpenStack features
Control APIs for managing VM images and setups
Integration to multiple cloud subsystems
User interface, dashboard
OpenStack Components
Quantum-network, Nova-compute, Keystone-identity
Cinder-block storage, Swift-object storage
Glance-image service
Savanna-Hadoop service
What does it provide
OpenStack is a key higher-level components in our Cloud solution
of today
A Cloud Framework like Openstack is usually required
in all real Cloud deployments
OpenStack
Agenda
Key Components and Architecture
Main Architecture
OpenStack
OpenvSwitch
QEMU/KVM
Virtualization Problems and Solutions
Further Work on the Topic
Hypervisor Pain Points
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
incl. HW acceleration
PIC
Guest VM
Application
HW-independent
acceleration API
Guest Network
Throughput
Real-time
Latency
Memory Access Speed
NIC Memory
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
incl. HW acceleration
PIC
Guest VM
Application
HW-independent
acceleration API
NIC Memory
Security and
Isolation
Multi-OS
support
Forward
portability
Hypervisor Pain Points
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
incl. HW acceleration
PIC
Guest VM
Application
HW-independent
acceleration API
NIC Memory
Full-stack
Management
Interfaces
Custom HW
adaptation
Hypervisor Pain Points
Background: Real-Time / Context Switching
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
PIC
Guest VM
Application
NIC
1) The interrupt event will enable a
thread for running, usually
invoking the host-side scheduler
first
2) The scheduler in the guest-side
will get activated to actually
thread inside the guest domain
3) The application running in the
thread will process the event
-> the critical components are the
context switching time and the
RT latency related to switching to
the right thread
Scheduler
ISR
Scheduler
Use-case: Real-Time / Context Switching
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
PIC
Guest VM
Application
NIC
Scheduler
ISR
Scheduler
Video Stack 1) The streaming video protocol
stack and endpoint have to
process packets at a certain
continuous rate to avoid dropping
the connection or having low
quality picture transfer
2) The application running the stack
is dependant on the scheduling
mechanism on the guest and
host
3) The timer interrupts are usually
controlled by the HW and are
used to control the timing,
sometimes a network
synchronized time source is used
Page 16
Solution: Real-Time / Context Switching
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
PIC
Guest VM
Application
NIC
Scheduler
ISR
Scheduler
1) Enhance latency response by
integrating Real-Time Linux to
the hypervisor
2) Improve the context switching
response by customizing the IPI
interrupt handling for the VM
guest
Video Stack
Page 17
Background: Network Throughput
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
Guest VM
Application
NIC
Guest Network
Throughput
Host Network
Throughput
1) The application inserts data into
the OS mechanisms that control
the HW packet transmission
inside the guest VM
2) The network handler in the guest
OS will feed the data to the
virtual interface (for example
virtio)
3) The host OS will handle
transmission from the virtual
interface to the actual HW,
possibly multiplexing the HW IP
interface
-> the critical components are the
propagation time from the guest
to the host and the effectiveness
of the host IP stack
Use-Case: Network Throughput
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
Guest VM
Application
NIC
Guest Network
Throughput
Host Network
Throughput
Control Plane
Server1) A control plane server application
is used to provide large amount
2) It uses standard networking to
send data from inside a
virtualized guest OS, which is
used to provide load-balancing
and failover facilities in the total
system.
3) The packet traffic is routed
through the guest and host OS
and if affected by delays at each
of the points, therefore affecting
the total capacity of the server
Solution: Network Throughput
Hardware
Tracing,
Logging,
Security
functionality
Kernel Layer
KVM/QEMU
Guest VM
Application
NIC
Guest Network
Throughput
Host Network
Throughput
Control Plane
Server1) Implement packet handling on
the host to bypass userspace
and enter the kernel or dedicated
HW directly
Eliminate processing for
traffic copy
Eliminate context transitions
2) Implement control mechanisms
for prioritizing and balancing
traffic in case of multiple VMs
Allow better QoS capabilities
for higher-priority functions,
like streaming data
Several Cloud users especially targeting embedded-
type deployments have significant SW legacy
HW consolidation of legacy OSs
Multi-version support for forward portability
future versions of the platform
Issues faced
Real-time
IO performance
However, usually new HW can do much better in relation..
Multi-OS Support
Hypervisor and Guest-to-Guest vulnerabilities are seen
as new fruitful ground for hackers
Multi-tenancy
Multi-node consolidation
Guests of lesser importance can be used to gain access
to the hypervisor and the more important Guest
domains
Hypervisor priviledge escalation
Careful management of CVEs needed
SELinux - sVirt
Not only exploits, also important data must not be
visible to non-authorized Guests
Packet traffic
Memory, IO
Security and Isolation
Key usability characteristic of a Cloud solution is the
accessibility to the provisioning and monitoring
systems
Custom GUIs for end solutions
Need access to APIs exposed by underlying facilities
Monitoring Solutions
Hypervisor and Guest logging
OpenStack interface adaptation
OpenStack plugin adapation
Libvirt as a key underlying facility
Full-Stack Management Interfaces
Agenda
Key Components and Architecture
Main Architecture
OpenStack
OpenvSwitch
QEMU/KVM
Virtualization Problems and Solutions
Further Work on the Topic
Co-operative Scheduling
Paravirtualized
Easy Adaptation of Applications
Across HW and Middleware
Adaptation layer for DPDK, SE, LWE, RTE etc
Performance and Manageability
OpenStack
Value-add-solutions
The Way Forward
Top Related