Newark, New Jersey 07102-5310 David I. Berl Christopher J ...
1 Chapter 12: Green Data Centers Yan Zhang and Nirwan Ansari Advanced Networking Laboratory New...
-
Upload
caleb-sawyer -
Category
Documents
-
view
217 -
download
2
Transcript of 1 Chapter 12: Green Data Centers Yan Zhang and Nirwan Ansari Advanced Networking Laboratory New...
1
Chapter 12: Green Data Centers
Yan Zhang and Nirwan AnsariAdvanced Networking LaboratoryNew Jersey Institute of Technology
Newark, NJ 07102
HANDBOOK ON GREEN INFORMATION AND COMMUNICATION SYSTEMS
2
Power Consumption of Data Centers
In 2007, Environmental Protection Agency (EPA) Report to Congress on Server and Data Center Energy Efficiency assessed trends in the energy usage and energy costs of data centers and servers in U.S.
Based on the power consumption of data centers in U.S. from year 2000 to 2006, the power consumption of data centers is predicted under five different scenarios: Two baseline prediction scenarios: historical trend scenario,
current efficiency trend scenario. Three energy-efficiency scenarios: improved operation
scenario, best practice scenario, and state-of-the-art scenario.
3
Power Consumption of Data Centers This prediction was performed with the total power consumption of
the installed base of servers, external disk drivers, and network ports in data centers multiplied by a power overhead factor caused by the power usage of power distribution and cooling infrastructure in data centers.
Historical trend: simply estimates the power consumption trends based on the observed power usage from year 2000 to 2006.
Current efficiency trend: estimates the power usage trajectory of U.S. servers and data centers by considering the observed efficiency trends for IT equipment and site infrastructure systems.
Data centers and servers in U.S. consumed about 61 billion kWh in 2006 for a total electricity cost of about $4.5 billion.
The energy use of data centers and servers in 2006 was more than doubled the electricity that was consumed by data centers in 2000.
It is estimated the energy usage of data centers could nearly double again in 2011 to more than 100 billion kWh with historical and current efficiency trends.
Improved operation trend: utilizes any essentially operational technologies requiring little or no capital investment to improve energy efficiency beyond “current efficiency trends”.
Best practice trend: adopts more widespread technologies and practices in the most energy-efficient facilities in operation.
State-of-the-art trend: maximizes the energy efficiency of data centers using the most energy-efficient technologies and best management practices available today.
4
Energy Efficiency of Data Centers
APC White Paper #6 (Ref [2]) investigated the total cost of ownership (TCO) of physical data center infrastructure, and found that the cost of electrical power consumption contributed to about 20% of the total cost.
Numerous studies have shown (Ref [3][4]): Data center servers rarely operate at full utilization. The average server utilization is often below 30 percent of the
maximum utilization in data centers.
At low levels of workload, servers are highly energy-inefficient.
5
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
Utilization (percent)
Ser
ver
Pow
er U
sage
(pe
rcen
t of
pea
k)
0 10 20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
Ser
ver
Ene
rgy
Eff
icie
ncy
(per
cent
age)
Power (typical)
Energy efficiency (typical)Power (proportional)
Energy efficiency (proportional)
Energy Efficiency of Data Centers
Server power usage and energy efficiency at varying utilization levels, from idle to peak performance (L. A. Barroso and U. Hölzle, “The Case for Energy-Proportional Computing,” Computer, 40(12):33–37, Dec. 2007).
Typical servers: consume about half of its full power at the idle state.
Power proportional servers: consume about half of its full power at the idle state.
6
Energy Efficiency of Data Centers
A typical data center power usage (adapted from M. Ton, B. Fortenbery, and W. Tschudi, “DC Power for Improved Data Center Efficiency,” http://hightech.lbl.gov/documents/DATA_CENTERS/DCDemoFinalReport.pdf, Mar. 2008).
7
Energy Efficiency Metrics for Data Centers
In order to quantify the energy efficiency of data centers, several energy efficiency metrics have been proposed to help data center operators to improve the energy efficiency and reduce operation costs of data centers: Power usage effectiveness (PUE) and data center infrastructure
efficiency (DCiE)
Data Center energy Productivity (DCeP)
Datacenter Performance Per Energy (DPPE)
Green Grid Productivity Indicator
8
Power Usage Effectiveness (PUE) The most commonly used metric to indicate the energy
efficiency of a data: PUE and its reciprocal DCiE. PUE definition:
PUE=1.0 implies: there is no power overhead and all power consumption of the data center goes to the IT equipment.
PUE measures the total power consumption overhead caused by the data center facility support equipment, including the cooling systems, power delivery, and other facility infrastructure like lighting.
Total Power Consumption of a Data CenterPUE =
Total Power Consumption of IT Equipment
1 Total Power Consumption of IT EquipmentDCiE = =
PUE Total Power Consumption of a Data Center
9
Power Usage Effectiveness (PUE)
The average data center PUE in the US in 2006 is 2.0, implying that one Watt of overhead power is used to cool and deliver every Watt to IT equipment (Ref[1]).
It also predicts that “state-of-the-art" data center energy efficiency could reach a PUE of 1.2 (Ref [66]).
Google publishes quarterly the PUE results from data centers with an IT load of at least 5MW and time-in-operation of at least 6 months (Ref [67]). The twelve-month, energy-weighted average PUE result
obtained in the first quarter of 2011 is 1.16, which exceeds the EPA's goal for state-of-the-art data center efficiency.
10
Data Center energy Productivity (DCeP)
Energy efficiency and energy productivity are closely related to each other. Energy efficiency focuses on reducing unnecessary power
consumption to produce a work output. Energy productivity of a data center measures the quantity of
useful work done relative to the amount of power consumption of a data center in producing this work.
DCeP allows the continuous monitoring of the productivity of a data center as a function of power consumed by a data center.
DCeP metric tracks the overall work product of a data center per unit of power consumption expended to produce this work.
Useful Work ProducedDCeP =
Total Data Center Power Consumed Producing this Work
11
Datacenter Performance Per Energy (DPPE)
DPPE evaluates the energy efficiency of data centers as a whole. The DPPE metric indicates data center productivity per unit energy.
DPPE defines four sub-metrics: IT Equipment Utilization (ITEU) ITEE (IT Equipment Energy Efficiency) PUE GEC (Green Energy Coefficient)
These four sub-metrics reflect four kinds of independent energy-saving efforts, and are designed to prevent one kind of energy-saving effort from affecting others.
12
Datacenter Performance Per Energy (DPPE)
ITEU It measures the degree of energy saving by efficient operation
of IT equipment through virtual techniques and other operational techniques.
ITEE It is defined as the ratio of the total capacity of IT equipment to
the total rated power of IT equipment. This metric aims to encourage the installation of equipment
with high processing capacity per unit electric power in data centers to promote energy savings.
Total Measured Power of IT EquipmentITEU =
Total Rated Power of IT Equipment
Total IT Equipment CapacityITEE =
Rated Power of IT Equipment
13
Datacenter Performance Per Energy (DPPE)
PUE: It indicates the power saving for data center facilities. The less power consumption of facility infrastructure, the
smaller the value of PUE.
GEC: It is defined as the ratio of the Green Energy produced and used
in a data center to its total power consumption. The value of GEC becomes larger if the production of non-CO2
energy is increased in a data center.
DPPE Considering the definitions of the above four sub-metrics, DPPE
incorporates these four sub-metrics and can be expressed as a function of them as follows:
Green EnergyGEC =
Total Power Consumption of a Data Center
1 1DPPE = ITEU ITEE
PUE 1-GEC
Total Power Consumption of a Data CenterPUE =
Total Power Consumption of IT Equipment
14
Green Grid Productivity Indicator
Green Grid Productivity Indicator is a multi-parameter framework to evaluate overall data center efficiency.
Through the use of a radial graph, relevant indicators such as DCiE, data center utilization, server utilization, storage utilization, and network utilization can be quickly, concisely and flexibly emerged to provide organizational awareness.
How it works: Set up the target value for each indicator. Plotting the peak and average values of each indicator during
the period of monitoring, together with their target and theoretical maximum values on a radial graph.
• Assess how well the data center resources are utilized, • Check if the business targets are achieved visually and quickly, • Figure out how to spend their efforts to maximize the benefits.
15
Green Grid Productivity Indicator
Examples of using the Green Grid indicator tool. (adopted from The Green Grid. “The Green Grid Productivity Indicator,” http://www.thegreengrid.org/~/media/WhitePapers/White_Paper_15_-TGG_Productivity_Indicator_063008.pdf?lang=en).
16
Techniques to Improve Energy Efficiency of Data Centers
IT Infrastructure Improvements Servers and Storages Network Equipment
Power DistributionSmart Cooling and Thermal Management Power Management Techniques
Provisioning Consolidation Virtualization
Others
17
IT Infrastructure Improvements
Approximately 40% - 60% power consumption of a data center is devoted to IT infrastructure, which consists of servers, storage, and network equipment
Servers and Storages CPU
Dynamic Voltage/Frequency Scale (DVFS): Dynamic Voltage/Frequency Scale (DVFS): 23% - 36% energy savings.
memory and disk power shifting (Ref [14]): re-budget the available power between processor power shifting (Ref [14]): re-budget the available power between processor
and memory. and memory. Power shifting is a threshold-based throttling scheme to limit the number of operations performed by each subsystem during an interval of time, but power budget violations and unnecessary performance degradation may be caused by improper interval length.
Mini-rank (Ref [16]): an adaptive DRAM architecture to limit power consumption of DRAM by breaking a conventional DRAM rank into multiple smaller mini-ranks with a small bridge chip.
Dynamic Rotations per Minute (DRPM) (Ref [17]): a low-level hardware-based technique to dynamically modulate disk speed to save power in disk drives since the slower the disk drive spins the less power it consumes.
18
IT Infrastructure Improvements
Energy proportional systems PowerNap (Ref [19]):
• Attune the server power consumptions to server utilization patterns.
• Transit rapidly between a high-performance active state and a minimal-power nap state in response to instantaneous load.
• PowerNap can be modeled as an M/G/1 queuing system.
19
IT Infrastructure Improvements
Network Equipment: switches, routers, wireless access points. Sleeping mode
• Transit into the low-power sleep mode when no transmission is needed, and return back to the active mode when transmission is requested.
• The transition time overhead of putting a device into and out of the sleep mode may reduce energy efficiency significantly.
Rate-adapting • The lower the line-speed is, the less power the devices
consume. • Adapt the transmission rate of network operation to the offered
workload.• Speed negotiation is required in the rate-adaption scheme for
both of the transmission ends.
20
Power Distribution
Current typical power delivery systems for data centers still use alternating current (AC) power: Distributed from utility to the facility, and is then stepped
down via transformers and delivered to uninterruptible power supplies (UPS).
Several levels of power conversion exist in both data center facilities and within IT equipment that results in significant electrical power losses, including power losses in UPS, transformers, and power line losses.
DC power distribution system has been demonstrated and evaluated for data centers (Ref [32]).
21
Smart Cooling and Thermal Management
Most data centers use liquid cooling for computer room air conditioning (CRAC).
Rack-level liquid-cooling solutions bring chilled water or liquid refrigerant closer to the servers. Rear-door liquid cooling Sealed rack liquid cooling In-row liquid-coolers Overhead liquid-coolers
Data center liquid cooling techniques tend to use naturally-cooled water, like lake or sea water Electrical savings by eliminating or reducing the need
for water chillers in data centers.
22
Smart Cooling and Thermal Management
The predominant air cooling scheme for current data centers is to use the CRAC units and an under-floor cool air distribution system.
23
Power Management - Provisioning
Provisioning is an effective solution to reduce the power consumption by turning off the idle servers, storages, and network equipment, or by putting them into a lower power mode. An adaptive dynamic server provisioning technique (Ref [49])
• Effective to dynamically turn on a minimum number of servers required to satisfy application specific quality of service and load dispatching.
• Tailored for long-lived connection-intensive Internet services..
A power-proportional cluster (Ref [50]) • Consists of a power-aware cluster manager and a set of
heterogeneous machines.• Uses currently available energy-efficient hardware, mechanisms for
transiting in and out of low-power sleep states, and dynamic provisioning and scheduling to minimize power consumption.
• Especially tailored for short lived request-response type of workloads.
24
Power Management - Provisioning
Sierra (Ref [53]) A power-proportional, distributed storage system. Turning off a fraction of storage servers during trough traffic
period. Utilizing a set of techniques: power-aware layout, predictive
gear scheduling, and a replicated short-term store, to maintain data consistency and fault-tolerance as well as system performance.
Rabbit (Ref [54]) A power-proportional distributed file system Provides ideal power-proportionality for large-scale cluster-
based storage and data-intensive computing systems by using a new cluster-based storage data layout.
Rabbit can maintain near ideal power proportionality even with node failures.
25
Power Management - Provisioning
ElasticTree (Ref [55]) Dynamically adjust the set of active network elements, links
and switches, to satisfy changing data center traffic loads. Given the data center network topology, the traffic demand
matrix, and the power consumption of each link and node, ElasticTree minimizes the total power consumption of a data center by solving a capacitated multi-commodity cost flow (CMCF) optimization problem.
Urja (Ref [56]) A network wide energy monitoring tool Integrated with network management operations to collect
configuration and traffic information from live network switches and to accurately predict their power consumption.
26
Power Management - Consolidation
Power savings with application consolidation Application consolidation in cloud computing (Ref [57]):
• The energy performance trade-offs for consolidation.• The application consolidation problem can be modeled as a
modified bin-packing problem, and the optimal points exist.
Generic application-layer energy optimization (Ref [58]):• Guides the design choices by using energy profiles of various resource
components of an application.
Intelligent data placement and/or data migration can be used to save energy in storage systems. Hibernator (Ref [59]):
• A disk array energy management system.• Several techniques to reduce power consumption while maintaining
performance goals, including disk drives that rotate at different speeds and migration of data to an appropriate-speed disk drive.
27
Power Management - Virtualization
Effective to enhance server utilization, consolidate servers and reduce the total number of physical servers. Power consumption caused by servers is reduced. Cooling requirement should also be reduced
comparably. Effective to build energy proportional storage systems.
Sample-Replicate-Consolidate Mapping (SRCMap) (Ref [61])• A storage virtualization solution for energy-proportional storage.• Consolidating the cumulative workload on a minimal subset of
physical volumes proportional to the I/O workload intensity. GreenCloud (Ref [62])
• Enabling comprehensive online monitoring, live virtual machine migration, and virtual machine placement optimization to reduce data center power consumption while guaranteeing the performance goals..
28
Others
Energy-aware routing (Ref[63]) An energy-aware routing optimization model. The objective is to find a route for a given traffic
matrix that minimizes the total number of switches. The proposed energy-aware routing model is NP-
hard, and a heuristic algorithm was required to solve the energy-aware routing problem.
Energy proportional datacenter network architecture (Ref[64]) A flattened butterfly data center topology is
inherently more power efficient than the other commonly proposed topology for high-performance data centers.
29
Conclusions Power consumption is a central critical issue for data
centers. As reported in 2005, the electricity usage of data centers has
been almost doubled from 2000 to 2005. The electricity cost accounts for about 20% of the total cost of
data centers.
Numerous studies have shown that the average server utilization is often below 30% of the maximum utilization in data centers.
To quantify the energy efficiency of data centers, several energy efficiency metrics have been proposed: PUE, DCiE, DCeP, DPPE, and Green Grid Productivity Indicator.
Techniques to improve energy efficiency of data centers.
30
Thanks for your attention!