Revolutionizing Data Center Efficiency McKinsey & Company UPTIME INSTITUTE SYMPOSIUM.
Modeling Rack and Server Heat Capacity in a Physics Based ... · Uptime Institute 2012 Data Center...
Transcript of Modeling Rack and Server Heat Capacity in a Physics Based ... · Uptime Institute 2012 Data Center...
Modeling Rack and Server Heat Capacity in a Physics Based Dynamic
CFD Model of Data Centers
Sami Alkharabsheh, Bahgat Sammakia 10/28/2013
ES2 Vision
2
Toward a full physics-based experimentally verified 3D
computational fluid dynamics model for data centers
To create electronic systems that are self sensing and regulating, and are optimized for energy efficiency at any desired performance level
Project Vision
Outline
Introduction Physics Based Steady State Baseline Model CRAC model
Server model
Tile model
Dynamic Model- Server Heat Capacity Effect Server level model
Room level model
Case studies
Conclusions and Future Work
3
Introduction
EPA (2007):1.5 % of total U.S. electricity consumption in 2006. (Total cost of $4.5 billion)
Datacenter Dynamics (2012) Global Census : power requirements grew by 63% globally to 38 GW from 24 GW in 2011.
Uptime Institute 2012 Data Center Industry Survey: PUE>1.8 for more than 55% of data centers
4
0%
2%
4%
6%
8%
10%
12%
14%
2.5
<=
2.4
-2.4
9
2.3
-2.3
9
2.2
-2.2
9
2.1
-2.1
9
2.0
-2.0
9
1.9
-1.9
9
1.8
-1.8
9
1.7
-1.7
9
1.6
-1.6
9
1.5
-1.5
9
1.4
-1.4
9
1.3
-1.3
9
1.2
-1.2
9
1.1
-1.1
9
1.0
9>=
Re
spo
nse
pe
rce
nta
ge
PUE
M. Iyengar and R. Schmidt, “Energy Consumption of Information Technology Data Centers”, 2010.
M. Stansberry and J. Kudritzki, “Uptime Institute 2012 Data Center Industry Survey,” Uptime Institute, 2013.
IT HVAC Cooling
Others
Nature of Problem
5
Solutions for improving the energy efficiency in data centers have been isolated
• Performance is not proportional to power
• Server overprovisioning is a common practice
• In real time, cooling is
difficult to control due to long lag times
•Complexity of transport in data centers
•Overprovisioning is commonly used for safe operation
Cooling Power
System-level and holistic solutions are a MUST
Fromtimes.com treehugger.com
Bench Mark Numerical Model
6
Raised Floor
Rack
Perforated tile
CRAC
Parameter Value
Room size 6.05 m x 13.42 m x 3.65 m
Plenum depth 0.6 m
Tile perforation ratio 50%
Perforated tiles area 0.61 m x 0.61 m
CRAC fan speed 100%
CRAC Model
Based on manufacturer data
Liebert 114D CW
Operating fan curve is obtained from the manufacturer, Liebert Consulting
The CRAC model is calibrated such that the flow rate can be predicted accurately at different operating pressures
7
* Alkharabsheh et al. “Utilizing Practical Fan Curves in CFD Modeling of Data Centers,” SEMITHERM2013.
Emersonnetworkpower.com
0 0.5 1 1.5 2 2.5 3 3.5 4
x 104
0
0.5
1
1.5
2
2.5
3
3.5
Flow rate (CFM)
Sta
tic p
ressu
re (
in. H
2O
)
CR
AC
in
tern
al re
sis
tan
ce
Calibrated operating point
Uncalibrated operating point
Server Model
8
2U server for
testing
A standard testing procedure following the AMCA 210-99 guidelines are used to measure the pressure fan curves
9 RU server simulators (load banks) and a 2 RU commercial server are tested
The measured fan curves include the internal resistance of the server
The measured fan curve can be imbedded directly into the CFD
Flow bench
apparatus
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45-50
0
50
100
150
200
250
Flow rate (m3/s)
Sta
tic p
ressu
re (
Pa
)
2 RU server
9 RU load bank
Tile Model
The CFD tile model is validated using experimental data in Schmidt et al.*
The CFD tile model is modified to compensate for the momentum loss in the CFD flow resistance model
The CFD tile model is able to capture the tile flow distribution and can be used in room level simulations
9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15-0.1
0
0.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15-0.1
0
0.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15-0.1
0
0.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15-0.1
0
0.1
Tile
Airflow
rate
(m
3/s
)
Row A
Row B
Row D
Row C
Solid line: experimental data, Dashed line: CFD results *Experimental data: Schmidt et al, “Measurements and Predictions of The Flow Distribution Through Perforated Tiles in Raised-Floor Data Center,” InterPACK2001
Computerfloorpros.com
Steady State Room Level Simulations
In addition to affecting the power dissipation, the servers power scenario also the airflow pattern by operating
The room can be underprovisioned/ overprovisioned based on the servers power level
Several parametric studies can be conducted using this model
10
15 17.5 20.5 24 28.1 32.9 38.5 45
Tem
pera
ture
(C)
15 kW/ rack
20 kW/ rack
32 kW/ rack
* Alkharabsheh et al. “Numerical Steady State and Dynamic Study in a Data Center Using Calibrated Fan Curves for CRACs and Servers,” InterPACK2013
Simple Dynamic Model
The thermal capacity of the equipment is not taken into account
Complete CRAC failure simulated at 20 seconds
Supporting the CRAC blower with backup power provides the room with extra cooling and time that can be utilized in increasing the reliability of operation
11
0 10 20 30 40 50 60 70 80 9020
30
40
50
60
70
80
90
100
110
Time (s)
Inle
t te
mpera
ture
(degC
)
No backup power
Blower backup power
Failure
Critical temperature
Unused plenum cold air
* Alkharabsheh et al. “Numerical Steady State and Dynamic Study in a Data Center Using Calibrated Fan Curves for CRACs and Servers,” InterPACK2013
Server Heat Capacity
The server level CFD model is developed based on the lumped mass approximation
Experimental data is used to calibrate and validate the server level CFD model
An increase in the rate of change in temperature is observed at low values of heat capacity until instantaneous change in temperature is noticed when server heat capacity is completely neglected
12
0 100 200 300 400 500 600 700 800 9003
3.5
4
4.5
5
5.5
6
Time (s)
T
se
rve
r
Exp. data [*]
CFD model
0 100 200 300 400 500 600 700 8003
3.25
3.5
3.75
4
4.25
4.5
4.75
5
Time (s)
T s
erv
er
Exp. data [*]
CFD model
0 100 200 300 4003
3.5
4
4.5
5
Time (s)
T
se
rve
r
No HC
1% Cap.
10% Cap.
50% Cap.
100% Cap.
120% Cap.
150% Cap.
*Ibrahim et al., "Thermal Mass Characterization of a Server at Different Fan Speeds," ITHERM2012.
Room Level Model
The detailed rack model is capable of hosting the server model, blanking panels, leakage through the mounting rails, and internal supports
Each server consists of an experimentally characterized fan curve and thermally calibrated heated mass
Each rack is populated with twenty of the 2 RU servers
13
n=20
n=1n=2
Server
Blanking panel
Mounting rails
Raised
Floor
Rack
Perforated
tile
CRAC
Detailed rack model
Case I: Servers Shutdown
It is assumed that all the servers inside the modular data center are shutdown at time 20 seconds
Three different room level models are compared in this transient analysis
Including the servers heat capacity is crucial in dynamic modeling. However, the heat capacity of the rack chassis can be neglected without affecting the accuracy of the results and reducing the computational time
14
0 200
20
Time (s)
Pow
er
(kW
/rack)
0 500 1000 1500 2000 25000
0.2
0.4
0.6
0.8
1
Time (s)
Rack inle
t te
mpeatu
re
No HC
Servers HC Only
Servers & Racks HC
Where: sso
ss
TT
TTT
ˆ
Case II: Server Power Short Pulses
Fluctuations in the dissipated power is simulated in the form of 30 second pulses
The temperature increases immediately in the model if we ignore the heat capacity
The heat capacity damps down the effect of short duration power fluctuations on the inlet temperatures
15
30 120180 100010
15
Time (s)
Pow
er
(kW
)
0 200 400 600 800 10000
0.2
0.4
0.6
0.8
1
Time (s)
Rack A
1 inle
t te
mpera
ture
Temperature without HC
Temperature with HC
Conclusions and Future Work
Experimentally validated models of different data center components are developed
A steady state and dynamic, physics based, room level CFD model for a bench mark data center is developed
It is found that the heat capacity of the servers affects the rate of change in temperature significantly
The effect of rack frames heat capacity is found to be small and can be neglected in room level simulations
Future work will include adding cooling unit heat capacity
16
Acknowledgement
This material is based upon work supported by the National Science Foundation under Grants No.1134867 and CNS-1040666
17