Intel® Cloud Builders Guide to Cloud Design and Deployment ... · blueprints and a software...

Intel® Cloud Builders Guide to Cloud Design and Deployment on Intel® PlatformsPower Management & Security within Open Source Private Cloud with Intel & OpenStack

Audience and PurposeThis document is intended for cloud administrators and enterprise IT professionals deploying cloud solutions based on OpenStack*. The document describes the solution architecture of a private cloud based on OpenStack that includes several enhancements to security and energy efficiency that take advantage of Intel processor architecture, advanced technologies, and software components.

September 2011

Intel® Cloud Builders GuideIntel® Xeon® Processor-based ServersPower Management & Security within Open Source Private Cloud with Intel & OpenStack

Intel® Xeon® Processor 5500 Series

Intel® Xeon® Processor 5600 Series

2

Intel® Cloud Builders Guide: Power Management & Security within Open Source Private Cloud with Intel & OpenStack

Table of Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Extensions to OpenStack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Server Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Server Security Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Solution Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Test-bed Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Hardware Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Physical Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Power Management Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Use Case One: Real Time Server Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Pre-requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Steps for Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Use Case Two: Policy Based Resource Distribution using VM Migrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Pre-requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14


Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Use Case Three: Optimize Rack Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Pre-requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17


Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Enhanced Security Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Use Case One: Platform Attestation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Pre-requisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20


Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Things to Consider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Architectural Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3


Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Additional Usage Models under Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Group Level (Aggregated) Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Secure VM Provisioning and Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Storage Management and Support for Additional Hypervisors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

APPENDIX A: Server Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Intel Power Management Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

Intel® Intelligent Power Node Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

APPENDIX B: Intel® Trusted Execution Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

Executive SummaryTo help businesses take advantage of the economic benefits of cloud computing, Intel pursues a vision that centers on three themes essential for simplifying the design, deployment and operation of cloud computing solutions: federated, automated, and client-aware. Federated means communications, data, and services can move easily and securely within and across cloud computing infrastructures. Automated means that cloud computing services and resources can be specified, located, and securely provisioned with very little or zero human interaction. Client-aware means that cloud computing solutions are both aware of and able to adapt seamlessly to take advantage of the capabilities of the end user’s device optimizing application delivery regardless of the type of device in hand.

Intel executes on this vision by implementing open, standards-based cloud computing solutions through a variety of means including academic and industry research, the Open Data Center Alliance (ODCA), Intel® Cloud Builders, engagement with ISV and OEM partners, and contributions to key open source virtualization and cloud computing projects. As a rapidly evolving open source project, OpenStack is designed to support massively scalable cloud computing infrastructures. Its adoption by a broad range of enterprises and service providers makes the implementation of usage models on this infrastructure widely useful.

Recent customer surveys on cloud computing cite security and compliance as primary issues slowing the adoption of cloud computing. Comprehensive security requires an uninterrupted chain of trust from the application user interfaces to the underlying hardware infrastructure. Any gaps in this trust chain invite attack targets. Today, security mechanisms in the lower stack layers (for example, hardware,

firmware and hypervisors) are almost absent. Yet another area of focus for leading solution providers and enterprises deploying massive scale datacenters is energy efficiency. Even though servers have become much more efficient, packaging densities and power have increased much faster. As a result, power and its associated thermal characteristics have become the dominant components of datacenter operational costs.

This reference architecture (RA) addresses these security and energy efficiency issues through a solution architecture that illustrates several specific usage models implemented in the OpenStack framework. It meets the requirements for real-time power monitoring and capping capabilities, managing power consumption using Intel® Intelligent Power Node Manager (Intel Node Manager). We also describe the concept of a trusted compute pool (TCP), which is a collection of physical platforms known to be trustworthy, using Intel® Trusted Execution Technology (Intel® TXT) available with Intel® Xeon® processors. Much of this implementation required the development of new software mechanisms, extensions to the OpenStack APIs, and enhancements to core components of the OpenStack architecture. The RA describes these modifications to OpenStack at a high level of detail. For more details of this work, we encourage customers to contact Intel representatives engaged with the OpenStack community at design summits, conferences, and other forums.

IntroductionThe world’s largest cloud service providers have long been working with Intel to take advantage of the innovations in performance, energy efficiency, virtualization, and security that Intel is bringing to the silicon, server, and datacenter technologies. Given this

heritage and unique role in the industry, Intel was drawn to participate as the technical advisor of the Open Data Center Alliance, an end user organization delivering next generation data center and cloud requirements in an open, industry standard and vendor agnostic fashion. To transform the usage models defined by the Open Data Center Alliance and other end user requirements into real-world deployments, Intel partners with the industry through the Intel Cloud Builders Program. Furthermore, recognizing the pace of innovation sustained by open source projects, Intel is working on key open source virtualization and cloud computing projects to develop open source implementations for several usage models.

The Intel Cloud Builders program brings together leading systems and software solutions providers to provide practical guidance on how to deploy, maintain and optimize a cloud infrastructure. As a central resource for reference architectures and best practices, Cloud Builders helps businesses accelerate their cloud deployments.

Published by the Intel Cloud Builders program, cloud reference architectures, such as the one described in this document, include a combination of client, server, storage, or network hardware blueprints and a software architecture using a partner ISV software stack that clearly illustrates how to implement selected set of pre-defined use cases.

OpenStack is an open source project designed to provide a massively scalable cloud operating system running on industry standard hardware. The technology consists of a series of interrelated projects delivering various components for a cloud infrastructure in private or public clouds. The project is backed by a global community of developers from over 100 companies including researchers, service providers,

4


and enterprises. All of the code is freely available under the Apache 2.0 license, allowing anyone to run it, build it, or submit changes back to the project.

Intel is contributing to OpenStack to enable it to make more intelligent decisions about placing workloads on the physical infrastructure. Intel is enhancing low-level modules in OpenStack to expose various hardware characteristics – such as trust, power consumption, quality of service, etc. -- up to the cloud management software. Intel is also enhancing the cloud software to make policy-based decisions that are informed by these hardware characteristics. And finally, Intel is enabling the operator to monitor the OpenStack deployment more closely via an updated management interface. As a result, OpenStack takes better advantage of advanced hardware technologies in modern server platforms and service providers are able to use their physical infrastructure more efficiently.

The Open Source Private Cloud (OSPC) solution based on OpenStack* that Intel® IT has developed demonstrates how enterprises can secure and manage a private cloud based on open source software by taking advantage of Intel technologies such as Intel TXT and Intel Node Manager.

In addition to enabling OpenStack to exploit the advanced capabilities of Intel-based platforms, the web-based interface developed by Intel allows users and administrators to manage a virtualization environment based on multiple hypervisors such as Xen and VMware ESX.

Some of these enhancements include:

• EC2-compatible interfaces for public clouds

• User control with different user groups, AD/LDAP authentication integration

• Policy-based resource monitoring and management

• Implement OpenStack object storage solution (Glance and Swift)

• Enterprise standard guest OS image provisioning and build (Microsoft Windows* 2003, 2008, SUSE)

• Low level physical RAS indicator through IPMI channel

• Third party monitoring solutions integration (Ganglia, Nagios)

• VM lease, snapshot, and migration

• Logging and report

• Unattended deployment package for flexible installation

The rest of the paper will focus on the usage models around power management and security. Discussion on the remaining features of OSPC is out of the scope of this paper.

Extensions to OpenStack Intel solutions for open source private cloud have extended OpenStack in below areas, which are also represented in Figure 1:

Open source Xen hypervisor support and optimization

• Extended EC2 API

• New Intel Node Manager module running on OpenStack node for power management

• Modification to cloud controller and API modules to accommodate the new Intel Node Manager module

Server Power Management Server power consumption is often an afterthought in data centers. For example, in many facilities the utility bill is bundled with the overall building charge which reduces the visibility of the data center cost.

Even though servers have become much more efficient, packaging densities and power have increased much faster. As a

result, power and its associated thermal characteristics have become the dominant components of operational costs.

Power and thermal challenges in data centers include:

• Increased total operational costs due to increased power and cooling demands.

• Physical limitations of cooling and power within individual servers, racks, and data center facilities.

• Lack of visibility into actual real-time power consumption of servers and racks.

• Complexity of management components and sub-systems from multiple vendors with incompatible interfaces and management applications.

• These challenges to manage data centers can be translated into the following requirements:

• Power monitoring and capping capabilities at all levels of the data center (system, rack identification, and data center). What can be done at an individual server level becomes much more compelling once physical or virtual servers are scaled up significantly.

• Aggregation of the power consumed at the rack level and management of power within a rack group to ensure that the total power does not exceed the power allocated to a rack.

• Higher level aggregation and control at the row or data center level to manage power budget within the average power and cooling resources available.

• Optimization of productivity per watt through management of power at the server, rack, row, and data center levels to optimize TCO.

5


• Application of standards-based power instrumentation solutions available in all servers to allow management for optimal data center efficiency. Extension of instrumentation to enable load balancing or load migration based on power consumption, and close coupled cooling for the management of pooled power and cooling resources.

Intel® Xeon® processors starting from Intel® Xeon® 5500 series processors onwards support Intel Node Manager, which regulates power consumption through voltage and clock frequency scaling. Reducing the clock frequency reduces power consumption, as does lowering voltage. The scale of reduction is accomplished through a series of discrete steps, each with a specific voltage and frequency.

Server Security ManagementCloud architectures abstract the physical hardware from logical compute units consumed by the user. As virtualization proliferates throughout the data center, the IT manager can no longer point to a specific physical node as being the home to any one critical processes or data—VMs may move to satisfy policies for high-availability, or resource usage. Regulatory compliance for certain types of data has also become increasingly difficult to enforce. Public cloud resources usually host multiple tenants concurrently, increasing the need for isolated and trusted compute infrastructure.

In order to minimize security risks, it is essential to protect and validate the integrity of the infrastructure on an

ongoing basis. This requires implementing the right tools and processes for protecting and validating all compute resources. One approach is to establish a “root of trust” – where each server must have a component that will reliably behave in the expected manner, and contain a minimum set of functions enabling a description of the platform characteristics, and its trustworthiness.

The value of Intel TXT is in the establishment of this root of trust, which provides the necessary underpinnings for reliable evaluation of the computing platform and the platform’s level of protection. This root is optimally compact, extremely difficult to defeat or subvert, and allows for flexibility and extensibility to measure platform components during

Figure 1

6


the boot and launch of the environment including BIOS, operating system loader, and virtual machine managers (VMM). Given the current nature of malicious threats prevalent in today’s environment and the stringent security requirements many organizations employ, a system cannot blindly trust its execution environment.

Solution ArchitectureStarting from the bottom layer, we have the physical node layer, the cloud management framework layer and the application layer. The physical node layer consists of the physical servers that are being managed by OSPC. The cloud management framework layer is the cloud management core layer that works in conjunction with the node layer and the application layer. It operates with the node layer for low level monitoring data collection and provides RESTful APIs for the application layer. The application layer is the cloud final management cockpit presented to end users.

The physical node layer consists of different types of hypervisors including open source Xen & VMWare ESX, each hypervisor managing a bunch of virtual machines.

The cloud management framework layer consists of hardware management, VM management and high level cloud management functionalities.

• Hardware management monitors the physical node for low level monitoring and data collection of thermal, base board, and voltage through NM/IPMI.

• VM management consists of:

° VM ops, which exposes EC2 compatible APIs and admin APIs for all the VM operations

° Network management manages and configures elastic IP and network configuration

° Image Store (Glance) is to utilize OpenStack Glance components to manage VM images which store images in OpenStack Swift Component, local drive, or Amazon S3

° VM Monitor monitors disk usage, CPU usage, memory workload, network IO, and VM information on VM through Ganglia and Libvirt

° Security is for VM startup authentication through Intel TXT and OpenStack Key Pair authentication

° Storage management is to create, delete, detach, and attach storage volume through OpenStack Compute Volume or Glance

• High level cloud management consists of:

° Policy-based management to implement Migration, Storage, Voltage, Thermal, Power, CPU usage based policy management

° User management by User group integrates with AD/LDAP Authentication and IEM Access

° VM Lease management is to manage VM lease through OSPC Instance Expiration Monitor

° Report and Log management is to generate , store and show report/log through Nagios

° Monitoring Data collects via IPMI, Ganglia and libvirt, and stores in RRD database

° VM Migration, Snapshot and Failover is for high availability and flexibility in Cloud

The cloud service management Cockpit (application layer) invokes and gets data from cloud management framework layer through RESTful APIs. It consists of:

• Resource indicator showing the resource usage and workload for VMs, CPU, memory, thermal and storage UI

• User access control for user/user group management and handling AD/LDAP

and IEM authentication integration

• Policy based management for creating and maintenance of policies

• Infrastructure event management to manage the infrastructure event

• Config compliance service is to streamline SW version updates and compliance in the enterprise environment

• Status monitoring to monitor cloud, nodes, and VM status

• On-demand migration is the policy-based V2V migration

• Event logging is to log and show nodes and VM service events

• SW delivery and automation service to automatically provide software stack based on SAAS

7


Figure 2

8


Functionality Software Hardware Description

Cloud Controller SLES 11 SP1OpenStack Compute Cactus ReleaseGanglia

Intel® Xeon® CPU X5630 @ 2.53GHz with 1.5G RAM, 200GB SATA HDDIntel® Baseboard Management Controllers (BMC) enabled

Portal Server CentOS 5.5NagiosLiferay

Intel Modular Server System MFSYS25 (Clearbay)2 Intel® Xeon® CPU X5560 @ 2.80GHz with 12G RAM, 100GB SATA HDDIntel BMC enabled

Node Controller (3) SLES 11 SP1Xen 4.0.0 HypervisorOpenStack Compute Cactus ReleaseGanglia

Intel Modular Server System MFSYS25 (Clearbay)2 Intel Xeon CPU X5560 @ 2.80GHz with 12G RAM, 100GB SATA HDDIntel BMC enabled

Node Controller (2) SLES 11 SP1Xen 4.0.0 HypervisorOpenStack Compute Cactus ReleaseGanglia

Intel Xeon CPU X5630 @ 2.53GHz with 1.5G RAM, 200GB SATA HDDIntel BMC enabled

Node Controller with Intel Node Manager

SLES 11 SP1Xen 4.0.0 HypervisorOpenStack Compute Cactus ReleaseGanglia

Intel® Xeon® CPU E5620 @ 2.53GHz with 5G RAM, 200GB SATA HDDIntel® Intelligent Power Node Manager enabledIntel® BMC enabled

Node Controller with Intel TXT

Fedora 14Xen 4.0.0 HypervisorOpenStack Compute Cactus ReleaseGanglia

Intel® Core™ i7 CPU 870 @ 2.93GHz with 3.6G RAM, 90GB SATA HDDIntel BMC enabledIntel TXT enabled

Image Service Node SLES 11 SP1OpenStack Image Service Cactus ReleaseGanglia

Intel Xeon CPU X5630 @ 2.53GHz with 1.5G RAM, 200GB SATA HDDIntel BMC enabled

Test-bed Blueprint The test-bed is intended to provide a flexible environment for simulating those aspects of a commercial data center that are relevant cloud computing usage models.

Design Considerations

Features include:

• Intel Node Manager compliant systems along with Advance Configuration and Power Interface (ACPI) compliant power supply for real-time power monitoring.

• 1GbE and 10GbE networks for achieving optimal performance during virtual machine migrations.

Hardware Description

9


Physical Architecture

Installation and ConfigurationIntel is currently working on getting the required approvals to submit the OSPC solution to the open source community and this paper will be refreshed after the approval with the appropriate links to the source code and setup documents. In the meanwhile if you need any help please contact the Intel sales team or go to the discussion forum on the Intel® Cloud Builders forum.

Figure 3

10


Power Management Use Cases

Use Case One: Real Time Server Power Monitoring

Power monitoring is a critical capability that enables us to characterize workloads and identify opportunities to increase data center energy efficiency.

1. Purpose

Server power monitoring helps us find and manage data center hotspots and thereby reduce the chances of hardware failure. The analysis of the historical usage of power consumption data enables us to develop forecast models for data center planning. These forecast models are very accurate as they are based on the real-time data.

2. Pre-Requisites

Intel Node Manager enabled hypervisor hosts, which can be a combination of open source Xen 4.0.0 and VMware ESX 4.1.0.

3. Steps for Execution

• Log in to the management portal using administrative credentials.

11


• Click on the “Power” tab on the top.

• In the next screen, you have either the option to view the historical usage of power or choose the real time power monitoring option. In our case we would click on the “Power Monitor” option at the bottom on the screen.

• Choose the Intel Node Manager compliant server with in the cluster.

Click on "Power Monitor"

12


4. Results

• Real time power utilization of the server is shown.

13


Use Case Two: Policy Based Resource Distribution using VM Migrations

1. Purpose

Real-time power consumption allows us to perform power aware resource distribution. Virtual machines that run the workload can be relocated to optimize and rebalance power margins based on measurements. The virtual machines can be relocated from power constrained systems to un-constrained systems within the cluster or across different clusters for better system utilization and performance.

2. Pre-Requisites

Intel Node Manager compliant server with few VMs.


• Click on the “Policy” tab at the top.

• Create a new policy with the action of “migration” as shown below. Here we have created a policy that gets activated when the power utilization of the server chosen goes beyond the threshold value specified.

• Save the policy and ensure that it is enabled.

• Below screen shot shows the two VMs currently running on Server8.

14


• Simulate a workload on the node and monitor the power consumption.

15


4. Results

TThe VM(s) running on the power constrained server to an un-constrained system once the server goes beyond the threshold value. As you see in the below screenshot, both of the VMs have been migrated out of the server on to other unconstrained servers within the cluster.

VMs migrated from Server8 to flex-TR server

16


Use Case Three: Optimize Rack Density

1. Purpose

The collection of real-time power consumption data constitutes an essential capability for power monitoring. Without this data, the best approximation for server power usage comes from the manufacturer’s specifications. To use the nameplate numbers as a guidepost requires the allowance of a hefty safety margin. To honor the safety margin in turn leads to data center power over-provisioning and stranded power that needs to be allocated in case it is needed, but is very unlikely to be used. This situation results in over-provisioned data center power, overcooling of IT equipment, and increased TCO.

The availability of power monitoring data allows management by numbers, which tightly matches servers by power consumption to available data center power. The use case is useful in older data centers under-provisioned for power and in host settings with power quotas in effect.

In typical host data centers where the customers are allocated power quotas, the main goal is to optimize the rack utilization so as to place as many servers in a rack as the power limit allows in order to maximize the microprocessor without interlocked pipeline stages (MIPS) yield. The number of machines will be so large that all machines will likely need to operate under a permanent cap. However, the overall MIPS yield for the collection of machines will be larger than otherwise possible for any combination of machines running uncapped, but whose aggregate power consumption is still subject to the rack power quota.

2. Pre-requisites

To measure the power utilization of the servers in the rack over a period of time and capture the maximum power utilized at the peak workload.

3. Steps for execution

• Click on the “Policy” tab at the top.

• Create a new policy with the action of “power capping” as shown below. Here we have created a policy that ensures that the power usage of the server will not go beyond the threshold value specified. Ensure that this threshold value is little lower than the maximum power utilization of the server captured earlier. Setting the threshold value to a value lower than the maximum will allow us to demonstrate the power capping policy.

Note: In reality, the threshold value would be higher than the maximum power utilization to ensure that the workload performance is not impacted.

• Save the policy and ensure it is enabled.

17


• Initiate the workload on the server for which the policy is enabled so as to increase the power utilization.

18


4. Results

• The below screenshot shows that the power utilization of the server stays within the threshold limit set from the server, which is 185 watts. In the earlier use case we observed that without any power cap, the power utilization of the server went up to about 275 watts

Note: Please note that the Intel Node Manager always tries to honor the power cap setting. But in certain cases, where the workload demand is very high, it will try to come down to the lowest p-state possible, if not to the level requested.

Note: Since the current version of our management solution does not support group level power capping, we need to set power caps at the individual server level for all the servers within the rack to optimize the rack density.

19


Enhanced Security Use Case

Use Case One: Platform Attestation

1. Purpose

In today’s increasingly virtualized environment, security concerns are amplified due to the complication of security management through:

• Multi-tenancy, employed to increase density and efficiency in the data center, and

• Software trust requirements combined with physical abstraction.

The purpose of this usage model is to showcase how Intel TXT can help address these problems, through the creation of a pool of trusted hosts, each with Intel TXT enabled, and through the validation of the launch of the platform and hypervisor. This trusted platform ensures that even if one VM is compromised, the physical host is still trusted and other VMs that co-exist can continue to run without any concerns. This trusted platform assures that physical hosts can maintain trust and prohibit compromised VMs access to their resources, and that trusted nodes can be managed in a scalable fashion.

2. Pre-Requisites

One cluster that consists of hypervisor hosts with Intel TXT and Intel Node Manager capabilities.


• Complete the configuration as specified in Installation and Configuration.

• Login to the OSPC portal and go to the home page.

• Click on the “Security” tab at the top and choose the Intel TXT enabled server for which we need to verify the status of platform attestation. In our demo environment, Server11 is the one that is enabled for Intel TXT.

20


4. Results

The tree view of the OSPC home page displays a green icon next to the Server11 indicating that the Server11 is a trusted server, which is based on Xen 4.0.0.

Things to Consider

Architectural Considerations

1. Power Management

Power management should be considered after careful analysis of the workload performance under various power capping. As mentioned earlier, there are many usage models for which a power management solution would be very beneficial. At the same time, there are scenarios wherein power management may not be the right option. For example, if a highly-sensitive production workload is very processor intensive and the host is already highly utilized, the addition of a power cap would inadvertently affect the performance of the system.

2. Security

Security is one of the key considerations in server deployments, either virtualized or bare-metal. In a cloud deployment

scenario, from both the perspective of the service provider and consumer, it is highly recommended to use platforms that support Intel TXT, such as Intel Xeon processor 5600 series, along with supporting software platforms to create a trusted cloud environment that enjoys strong protection against compromise.

3. Hardware

A full discussion of processor and overall server performance considerations is beyond the scope of this paper. However, it is important to note that the performance of VMs that run in a virtualized platform is heavily influenced by factors of processor architecture and specific feature sets available in the processor. The use of high performance server processors equipped with virtualization and I/O support feature sets, such as Intel Xeon processor 5500 and 5600 series, which also supports Intel

Node Manager is strongly recommended. For more details on Intel® Virtualization technologies please refer to www.intel.com/technology/virtualization/ and download.intel.com/business/resources/briefs/xeon5500/xeon_5500_virtualization.pdf.

Status of Server11 changed to Trusted

21


http://www.intel.com/content/www/us/en/virtualization/virtualization-xeon-5500-superior-hardware-platform-for-server-virtualization-brief.html





Additional Usage Models Under Development

1. Group Level (Aggregated) Power Management

In this paper we demonstrated power monitoring and capping at individual server levels. In the future versions, there would be capabilities to both monitor and cap at an aggregated or group level. Here the user would put a power cap say at the rack level, which then would be distributed internally to all the servers within the rack. The power allocation to the individual servers would depend upon the historical usage of the servers.

2. Secure VM Provisioning and Migration

In the current version of OSPC, only the trust status of the platform is being shown. Going ahead the capability for the end user to request for a trusted host to run their VM would be added. Internally OSPC would provision the new VM on a trusted host and in case the VM needs to be migrated either because of maintenance or resource distribution, it would be migrated on to another trusted host.

3. Storage Management and Support for Additional Hypervisors

Future releases of OSPC would have additional capabilities for storage management for VMs and would be supporting additional hypervisors including Microsoft Hyper-V* and KVM.

GlossaryIntel® Intelligent Power Node Manager (Intel Node Manager): Intel Node Manager resides on Intel Xeon processor 5500 series server (and later) platforms. It provides power and thermal monitoring and policy based power management for an individual server. Capabilities are exposed through standard intelligent platform management interface (IPMI) interface from supported Baseboard

Management Controllers (BMC). This requires an instrumented power supply such as PMBus*.

Intel® Trusted Execution Technology (Intel TXT): a hardware solution that validates the behavior of key components within a server or PC at startup.

Authenticated Code Modules (ACM): Platform-specific code that is authenticated to the chipset and that is executed in an isolated environment within the CPU. This term is also used to denote Authenticated Code Mode that is a trusted environment enabled by an AC Module to perform secure tasks.

Measured Launch Environment (MLE): An environment that is measured and launched as a result of the GETSEC [SENTER] instruction. This can be an operating system, virtual machine manager, or any trusted code that supports Intel TXT.

Trusted Platform Module (TPM) 1.2 (third party silicon): A hardware device defined by the Trusted Compute Group that provides a set of security features used by Intel TXT.

Secure Initialization (SINIT): A trusted process that measures, validates, and launches an MLE.

Safer Machine eXtensions (SMX): The capabilities added to Intel processors that enable Intel TXT.

Trusted Computing Group (TCG): Industry initiative for advancing computer security (http://www.trustedcomputinggroup.org)

Virtual Machine Extensions (VMX): A set of processor instructions defined by Intel Virtualization Technology that software uses to provide isolation and protection for virtual environments (part of VT-x).

Intel® Virtualization Technology for Directed I/O (Intel VT-d): Hardware support component of Intel Virtualization Technology for management of DMA and interrupts generated by I/O devices.

Intel® Virtualization Technology for Execution™ (Intel VT-x): A set of processor instructions (VMX) and capabilities defined by Intel Virtualization Technology that software uses to provide isolation and protection for virtual environments.

22


References1. Intel® Intelligent Power Node

Manager, http://www.intel.com/technology/intelligentpower/index.htm

2. Intel® Xeon® Processor 5500 Series, http://www.intel.com/itcenter/products/xeon/5500/index.htm

3. Intelligent Platform Management Interface, http://www.intel.com/design/servers/ipmi/ipmi.htm

4. PMBus*, http://pmbus.org/specs.html

5. Advanced Configuration & Power Interface, http://www.acpi.info/

APPENDIX A: Server Power Management

Intel Power Management Technologies

Micro-processors are possibly the most energy intensive components in servers and have traditionally been the focus of power management strategies. Emerging technologies such as solid state drives have the potential to significantly reduce power consumption and in

future, management of memory power consumption may be incorporated.

Intel Node Manage1 is designed to address typical data center power requirements such as described above.

Intel Node Manager is implemented on Intel® server chipsets starting with Intel Xeon processor 5500 series platforms2. Intel Node Manager provides power and thermal monitoring and policy based power management for an individual server and is exposed through a standards based IPMI interface3 on supported Baseboard Management Controllers (BMCs). Intel Node Manager requires an instrumented power supply conforming to the PMBus standard4.

Intel Intelligent Power Node Manager

Intel Xeon processors regulate power consumption through voltage and clock frequency scaling. Reducing the clock frequency reduces power consumption, as does lowering voltage. The scale of reduction is accomplished through a series of discrete steps, each with a specific voltage and frequency. The Intel Xeon

Figure 4: Intel Node Manager power management closed control loop

processor 5500 series can support 13 power steps. These steps are defined under the ACPI (Advanced Configuration and Power Interface)5 standard and are colloquially called P-states. P0 is nominally the normal operating state with no power constraints. P1, P2 and so on aggressively increase the power capped states.

Voltage and frequency scaling also impacts overall system performance, and therefore will constrain applications. The control range is limited to a few tens of watts per individual micro-processor. This may seem insignificant at the individual micro-processor level, however, when applied to thousands or tens of thousands of micro-processors typical found in a large data center, potential power savings amount to hundreds of kilowatt hours per month.

Intel Node Manager is a chipset extension to the BMC for supporting in-band / out-of-band power monitoring and management at the node (server) level. Some of the key features include:

23


• Real-time power monitoring

• Platform (server) power capping

• Power threshold alerting

The below figure shows the Intel Node Manager server power management closed control loop.

Keeping in pace with the changes in cloud computing, Intel is working on the next version of Intel Node Manager, where in there would be support for additional controls for better power management. These changes would support additional usage models for policy based power management.

APPENDIX B: Intel Trusted Execution TechnologyIntel TXT is a set of enhanced hardware components designed to protect sensitive information from software-based attacks. Intel TXT features include capabilities in the microprocessor, chipset, I/O subsystems, and other platform components. When coupled with an

Figure 5: Intel TXT components

enabled operating system, hypervisor, and enabled applications, these capabilities provide confidentiality and integrity of data in the face of increasingly hostile environments.

Intel TXT reduces the overall attack surface for individual systems and compute pools. Principally, Intel TXT provides a signature of the launch environment to enable a trusted software launch and to execute system software. The protection of the launch environment ensures that the cloud infrastructure as a service (IaaS) has not been tampered with. Additionally, security policies based on a trusted platform or pool status can then be set to restrict (or allow) the deployment or redeployment of VMs and data to platforms with a known security profile. Rather than reliance on the detection of malware, Intel TXT works because it builds trust into a known software environment and thus ensures that the software being executed hasn’t been compromised. This advances security

to address key stealth mechanisms used to gain access to parts of the data center in order to access or compromise information. Intel TXT works with Intel VT to create a trusted, isolated environment for VMs.

Intel TXT incorporates a number of secure processing innovations, including:

• Trusted extensions integrated into silicon (processor and chipset)

• Authenticated code modules (ACM): platform-specific code is authenticated to the chipset and executed in an isolated environment within the processor and the trusted environment (authenticated code mode) enabled by AC Modules to perform secure tasks.

• Launch control policy (LCP) tools

• Some of the required components for the Intel TXT secured platform are provided by third parties, including:

• • Trusted Platform Module (TPM) 1.2 (third party silicon)3: A hardware device defined by the Trusted Compute Group that stores authentication credentials in platform configuration registers (PCRs), which are issued by Intel Trusted Execution Technology

• Intel TXT-enabled BIOS, firmware, operating system, and hypervisor environments

24


Intel TXT Capabilities

The capabilities of Intel TXT include:

• Protected execution: Lets applications run in isolated environments so that no unauthorized software on the platform can observe or tamper with the operational information. Each of these isolated environments executes with the use of dedicated resources managed by the platform.

• Sealed storage: Provides the ability to encrypt and store keys, data, and other sensitive information within the hardware. This can only be decrypted by the same environment that encrypted it.

• Attestation: Enables a system to provide assurance that the protected environment has been correctly invoked and to take a measurement of the software running in the protected space. The information exchanged during this process is known as the

attestation identity key credential and is used to establish mutual trust between parties.

• Protected launch: Provides the controlled launch and registration of critical system software components in a protected execution environment.

Intel Xeon 5600 processor series support Intel TXT, which is designed to address such software-based attacks. For more information on Intel TXT, please visit http://www.intel.com/technology/security.

Intel TXT: Principle of Operation

Intel TXT works through the creation of a measured launch environment (MLE) that enables an accurate comparison of all the critical elements of the launch environment against a known good source. Intel TXT creates a cryptographically unique identifier for each approved launch-enabled component and then provides hardware-based enforcement mechanisms to block

Figure 6: How Intel TXT protects a virtualized environment

the launch of the code that does not match that which is authenticated. This hardware-based solution provides the foundation on which IT administrators can build trusted platform solutions to protect against aggressive software-based attacks.

Figure 6 illustrates two different scenarios. In the first, the measurements match the expected values, so the launch of the BIOS, firmware, and VMM are allowed. In the second, the system has been compromised by a root-kit hypervisor, which attempts to install itself below the hypervisor to gain access to the platform. In this case, the Intel TXT-enabled, MLE-calculated hash system measurements will differ from the expected value, due to the insertion of the root-kit. Therefore, the measured environment will not match the expected value and, based on the launch policy, Intel TXT could abort the launch of the hypervisor.

25


Disclaimers∆ Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. See www.intel.com/

products/processor_number for details.INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROP-

ERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or “undefined.” Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The infor-mation here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized er-rata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which

have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or by visiting Intel’s Web site at www.intel.com.Copyright © 2011 Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Xeon inside, and Intel Intelligent Power Node Manager are trademarks of IntelCorporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others.


Intel® Cloud Builders Guide to Cloud Design and Deployment ... · blueprints and a software...

Documents

Transcript of Intel® Cloud Builders Guide to Cloud Design and Deployment ... · blueprints and a software...