Deploying High-Density Pods in a Low-Density Data … High-Density Pods in a Low-Density Data Center...

Deploying High-Density Pods in a Low-Density Data Center

Revision 2

by Neil Rasmussen and Victor Avelar

The problem: unmanaged high density

3

The solution: high-density pods

4

Pod containment methods 6

Additional high-density podbenefits

12

In-house vs. vendor-assisted deployment

13

Real-time management of high-density pods

16

Conclusion 17

Resources 18

Appendix A 19

Click on a section to jump to it Contents

White Paper 134

Simple and rapid deployment of self-contained, high-density pods within an existing or new low-density data center is possible with today’s power and cooling technology. The independence of these high-density pods allow for predictable and reliable operation of high-density equipment without a negative impact on the performance of existing low-density power and cooling infrastructure. A side benefit is that these high-density pods operate at much higher electrical efficien-cy than conventional designs. Guidance on planning design, implementation, and predictable operation of high-density pods is provided.

Executive summary>

white papers are now part of the Schneider Electric white paper libraryproduced by Schneider Electric’s Data Center Science Center [email protected]


Schneider Electric – Data Center Science Center White Paper 134 Rev 2 2

High-density equipment such as blade servers, 1U servers, and multi-core, high-end servers provide more computing-per-watt compared to previous generation servers. However, when consolidated, this equipment requires concentrated power and cooling resources. Data center operators and IT executives are often uncertain about the capability of their existing data center and whether a new data center must be built to support higher rack densities. A simple solution exists that allows for the rapid deployment of high-density racks within a traditional low-density data center. A high-density pod, as illustrated in Figure 1, allows data center managers to support a mixed-density data center environment for a fraction of the cost of building an entirely new data center.

High-density pod

Hot/cool air circulation is

contained within the pod

HEAT ENERGY OUT – to heat rejection system

Low-density room

• A high-density “island” in the room• A “mini data center” with its own cooling• Thermally neutral or even positive to the rest of the room • Hot/cool air circulation is localizedwithin the pod by short air paths and/or physical containment

In this paper a high-density pod is defined as one or more rows of racks containing high-density equipment all clustered together with dedicated row-based cooling that is deployed as a unit. A high-density pod resides within the borders of a larger, low-density data center. The high-density pod is not the same as a high-density data center, which is a data center dedicated to supporting nothing but, or mostly, high-density racks. Managing for the deploy-ment and operation of a high-density data center is not the subject of this paper. High-density pod compared to “spreading out” strategy Although today’s IT equipment operates at high power density – that is, each individual server draws a high amount of power – this does not always mean such devices must be deployed in a high-density manner by packing them together in a rack. In fact, a popular strategy has been to spread out high-density servers by installing fewer in each rack. If the equipment is dispersed like this, the data center’s average power density will likely stay in the range that the data center was originally designed for. In this way, a variety of technical problems can be avoided.

Introduction

Figure 1 Basic concept of a high-density pod

> High density enables high efficiency In traditional data centers with room-based power and cooling, unmanaged high-density racks can cause destabilizing effects such as cooling inefficiency, loss of cooling redundancy, hot spots, thermal shutdown, and circuit overload. However, with today’s new power and cooling technologies, high-density racks offer an opportunity for dramatically increased efficiency and predictability, if deployed effectively and supported by “smart” row-based power and cooling. The high-density pods described in this paper provide a way to deploy high density while at the same time achieving increased overall data center efficiency by targeted, scalable, localized power and cooling.



However, the “spreading out” strategy may not be viable for a number of reasons: • Consumption of additional floor space, which may be difficult to justify or simply not

possible

• Executive management perception that partially-filled racks are wasteful

• Increased cabling costs (because of longer runs)

• Increased cost and difficulty of maintenance – cabling and mounting may be intertwined with other equipment in a non-standard manner, scattered throughout the room

• Reduced electrical efficiency of the data center, because cooling-system air paths are longer and less well targeted. The longer the air path in an uncontained system, the more chance there is for hot and cold air to mix. This mixing results in the lowering of the return temperature to the air conditioner which means the system will be less effi-cient in removing heat energy. For more information regarding the containment of air streams, see White Paper 135, Hot-Aisle vs. Cold-Aisle Containment for Data Centers.

For these reasons, it is expected that data enter operators will begin to deploy IT equipment at its full density capability – in pods – rather than try to stay within an overall room power density by spreading out the load. With new power and cooling technologies, there is now a significant efficiency entitlement from concentrating high-density equipment into pods. This paper assumes the choice has been made to deploy high-density IT racks in a low-density data center. Row-based cooling, as a technique to implement these high-density pods, is presented as a simple solution for addressing high-density power and cooling issues in both existing and new data centers. For more information regarding alternatives for deploying high-density equipment, including the option of spreading out IT equipment, see White Paper 46, Cooling Strategies for Ultra-High Density Racks and Blade Servers. Traditional data center design uses a raised floor to distribute cooling to low-density IT equipment (Figure 2a) and air streams are uncontained. However, when high-density equipment is randomly installed throughout a low-density data center the cooling stability is upset and hot spots begin to appear (Figure 2b). Data centers designed for low-density racks (typically 1-3 kW / rack) vary dramatically in construction. Ceiling heights, raised floor depths, room geometry, power distribution, and raised floor obstructions are all quite different. In addition, IT managers vary in how they define a high-density rack. This paper defines a high-density rack as 6 kW or higher. Regardless of which number is used to denote a high-density rack, the following deployment issues need to be considered:

The problem: unmanaged high density

Low-density room

Concentrated high-density IT equipment

Unpredictable cooling

Low-density room

Stable cooling

Figures 2a (left) and 2b 2a – Low-density data center 2b – High-density hot spots

Cooling Strategies for Ultra-High Density Rack and Blade Servers

Related resource White Paper 46

Hot-Aisle vs. Cold Aisle Containment for Data Centers


http://www.apc.com/wp?wp=135




• Delayed server deployment – uncertainty of knowing which rack can cool a newly provi-sioned server just adds to the already long delay by having to perform a cooling as-sessment

• Unplanned downtime – due to overloaded power distribution circuits or thermal shut-down of IT equipment

• Unpredictable cooling throughout the data center – no certainty that every high-density server will be properly cooled after every move, add, or change (see White Paper #121, Airflow Uniformity Through Perforated Tiles in a Raised-Floor Data Center).

• Loss of cooling redundancy – as more high-density racks are added, air conditioning units that were once redundant are now required to supply the concentrated airflows. Some subsystems are extremely impractical or costly to instrument for power consump-tion (for example, PDUs due to number of output connections, or switchgear)

Fortunately a solution exists that can neutralize these issues and is discussed in the following sections. Placing high-density racks in an isolated, standardized, and self-contained area of the data center provides a low cost, viable solution to the challenges mentioned above. This high-density pod avoids dependence on the unpredictable nature of raised floor cooling and would not require complex computational fluid dynamics (CFD) analysis prior to installation. Figure 3 illustrates three high-density pod implementation methodologies all of which are capable of supporting independent power distribution, UPS, and cooling systems. This “drop-in” solution eliminates the hot spots in Figure 2b by simply moving high-density equipment into the pod that contains dedicated row-oriented cooling units. The heat generated from the high-density IT equipment within this pod is rejected to the outdoors with no negative impact to the existing data center cooling system or the surrounding low-density IT racks. In fact, the pod acts as its own high-density data center within an existing low-density data center. This self-sufficient pod is, at a minimum, thermally “invisible” or, more likely, is a net exporter of cooling to the rest of the room.

The solution: high-density pods


localized within the zone

Low-density room

Uncontained HOT-AISLE containment

RACK containment

Three basic methods(Top view)

High-density zone

HEAT OUT to building’s heat rejection system

Figure 3 Isolated, self-sufficient high-density pod

Airflow Uniformity Through Perforated Tiles in a Raised-Floor Data Center





What is thermal “visibility”? Isolated, standardized and self-contained high-density pods operate on the idea of isolating server exhaust heat and directing all of that heat into the air conditioner intakes, where the air is then cooled before being redistributed to the front of the servers. By isolating both hot and cold air streams, the high-density pod, at a minimum, neutralizes the thermal impact that high-density IT racks would otherwise have on traditional low-density data centers. In other words, the pod is thermally “invisible” to the existing data center cooling system. Particularly with hot-aisle and rack containment methods, however, it is quite likely that the row-oriented cooling in the high-density pod will have a positive effect by actually adding cooling capacity to the rest of the room. Although this paper focuses on the cooling of high-density pods, it is also possible to power a pod with its own dedicated UPS and power distribution. This may be desirable in situations where the existing data center UPS is at capacity or is being phased out due to end-of-life or when targeted power availability is required for a specific pod. The system in Figure 4 integrates a cluster of high-density IT racks with a high-density row-based cooling system and high-density UPS and power distribution system in a pre-manufactured, pre-tested pod.

Integrated row-based air conditioners

IT rack

ITrack

ITrack

ITrack

Air conditioners return neutralized (ambient room temperature) air to the front of the racks

Hot air is exhausted to the hot aisle and returns to the back of the air conditioners

Row-based cooling architecture A row-based cooling architecture makes it possible to have a room-neutral high-density pod. Row-based cooling is an air distribution approach in which the air conditioners are dedicated to a specific row of racks. This is in stark contrast with room-based cooling where perimeter air conditioners are “dedicated” to the entire room. Row-based air conditioners may be installed above IT racks, adjacent to IT racks, or in combination. An example of a row-based air conditioner is shown in Figure 5.

Figure 4 Front-view of a standardized modular multi-rack high-density pod (no containment in this example)



While most facilities and IT personnel understand the basic idea behind high-density pods, they question how the pod can be “room neutral” in the midst of constant moves, equipment additions, and changes. Consi-dering their past experience with the variability and at times perplexing nature of raised-floor cooling, skepticism toward the long-term predictability of high-density pods is not surprising. Though raised floors and high-density pods are both governed by the same laws of fluid dynam-ics and thermodynamics, one major aspect sets them apart – standardization. If raised floors were standardized so that they all had the same depth, same dimensions, same under floor obstructions, same under floor airflow pattern, same CRAC locations, and same air leakage from tile cutouts, they could more easily be modeled in real time so as to predict their behavior using design and planning software tools. If this standardization existed, IT managers would be able to predict the cooling impact of adding a blade chassis to a particular rack and make rational decisions based on the prediction. However, these raised floor attributes by their very nature are customized and are not conducive to standardization. Furthermore, the variability of all these attributes would make real-time computational fluid dynamics (CFD) modeling nearly impossible in a typical data center. In contrast, high-density pods use standardized hot / cold aisle widths, rack height, and air path distances to the rack. Row-based cooling also eliminates the variability introduced by the raised floor. These simplifications make it possible to design predictable high-density pods using standardized tools. These design tools provide the confidence that any design will capture and neutralize the expected amount of hot exhaust air. For more information on the row-based cooling architecture, and how it compares to room-based cooling, see White Paper 130, The Advantages of Row and Rack-Oriented Cooling Architectures for Data Centers. Server exhaust heat can be diverted back to the air conditioners in three ways: uncontained, hot aisle containment, and rack air containment (see Figure 5). All of these methods leverage a row-based cooling concept (e.g., the air conditioner is brought within a few feet of the IT rack).

Pod containment methods

> Row-based cooling unitsCompared with the traditional room-oriented approach, the airflow paths of row-based air conditioners are shorter and much more predictable. In addition, all of the rated capacity of the air conditioner can be utilized, and higher power density can be achieved. At the same time, the usable capacity of the perimeter (room-based) cooling system increases and in some cases its cooling redundancy is restored to the original design as IT load is removed from this system and placed into the pod. Although not discussed in this paper, row-based cooling is also an effective method for entirely cooling small low-density data rooms (1-3 rows of racks).

The Advantages of Row and Rack-Oriented Cooling Architectures for Data Centers





1. Uncontained Uncontained pods rely on the standard layout and widths of the common hot aisle and cold aisle arrangement to keep hot and cold air streams from mixing. For this reason, uncon-tained pods depend on multiple racks in a row and are not effective in cooling stand-alone IT racks. The hot and cold aisles formed by rows of racks (and in some cases walls) are what isolate the hot and cold air streams as illustrated in Figure 6. The closer an IT equipment rack is to a row-based air conditionec, the greater the amount of exhaust air that is captured and cooled. As the distance between the IT rack and the row-based air conditioner increases in an uncontained system, the more the hot exhaust air mixes with the surrounding air in the data center. When to use this method: • When IT racks designated for the pod are moved and relocated frequently

• When IT racks are used from a variety of different vendors

Trade-offs: • More row-based air conditioners required at lower densities in order to properly capture

hot exhaust air from all IT racks.


localized within the zone

Low-density room

High-density zone

Three ways to create a room-neutral “island” in a low density room

1

2

3

Uncontained

HOT-AISLE containment

RACK containmentRoom-neutral “island”

in a low-density room

Figure 5 High-density pod containment methods

CR

AC

CR

AC

CR

AC

Rack Rack Rack Rack

REAR

FRONT

WALL or ROW to help form “hot aisle”

Hot aisle

Figure 6 High-density pod with no containment

> Importance of blanking panels Effective row-based cooling depends on the isolation of hot and cold air streams. If any of the vertical space in a rack is not filled by equipment, the gaps between equipment allow hot exhaust air to flow through the rack and to the front of equipment such as servers. This mixing between the hot and cold air streams reduces the effectiveness of row-based cooling. For more information see White Paper 44, Improving Rack Cooling Performance Using Airflow Management™ Blanking Panels (link in Resource section)



2. Hot aisle containment Hot aisle containment pods are identical to uncontained pods except for the fact that the hot aisle in every pair of rows is contained. The hot aisle becomes the hot exhaust channel by enclosing it with ceiling panels and a door at each end of the aisle (Figure 7). In addition, the racks’ rear doors are removed. The hot exhaust air is physically contained and unable to mix with the ambient data center air. A wall or another row of racks is required to form a cold aisle in order to isolate the cold supply air. When to use this method: • In cases where floor space must be conserved. This method is popular because it

consumes the same space as two rows of low-density racks.

• In data centers with hot aisle / cold-aisle layouts

Trade-offs: • Hot aisle containment panels increase capital cost

• Hot aisle containment may exceed work environment policies due to high temperature

• Incompatible with some types of cabling, power strips, labels, and other materials that are not rated for high temperatures

• Not possible with a single row of racks

• Authority having jurisdiction (AHJ) may require fire suppression in hot aisle

3. Rack containment Rack containment (also called rack air containment) is similar to hot aisle containment except that the hot exhaust air is contained using the back frame of the equipment racks and a series of panels to form a rear air channel. This channel can be attached to a single IT rack or to a row of racks (Figure 8). The panels used to create the hot exhaust air channel increase the depth of a normal rack by 20 cm (8 in). An optional series of front panels may be used on rack containment arrangements that require complete containment of hot and cold air streams as shown in Figure 9. This optional front containment adds an additional 20 cm (8 in) to the depth of the rack.

CR

AC

CR

AC

CR

AC

Rack Rack Rack Rack

CR

AC

CR

AC

CR

AC

Rack Rack Rack Rack

Hot aisleREAR

FRONT

FRONT

Containedhot aisle

Figure 7 High-density pod with hot aisle containment



When to use this method: • In cases where hot aisle containment is the preferred method, but a single odd row is

left uncontained

• When frequent access to and easy management of communication cables is required

• For complete isolation in cases such as stand-alone open data center environments or mixed layouts – only when optional front containment is used

• In wiring closets that lack any form of cooling, exposing high-density equipment to high temperatures – only when optional front containment is used

• When sound attenuation is required – only when optional front containment is used

Trade-offs: • Front and rear containment panels increase capital cost

• In a single rack configuration, cost increases substantially when cooling redundancy is required

Figure 8 High-density pod with rack containment C

RA

C

CR

AC

CR

AC

REAR

FRONT

Solid rear doors

Return air contained

RackRackRackRack

> Why NOT use containment? It may appear that containment would be the clear choice for any row-based cooling scenario. However, this is not always the case. With row-based cooling, containment is more important at lower densities, where the ratio of IT racks to air condition-ers is higher. The higher this ratio the greater the distance between IT racks and air conditioners, with more chance for hot exhaust air to “escape.” Higher densities, on the other hand, mean a lower ratio of IT racks to air conditioners, with shorter air paths and less chance for hot exhaust to escape – in this case, contain-ment is less essential because airflow is tightly targeted and tends to “behave” all by itself. In addition, there may be practical considerations that rule out containment, such as higher cost at certain rack power densities, company restrictions on hot work environments (i.e., a contained hot aisle), and incompatibility with existing racks.



An overall comparison of high-density pod methods is shown in Table 1.

CR

AC

CR

AC

CR

AC

Rack Rack Rack Rack

REAR

FRONT

Solid rear doors

Solid front doors

REAR

FRONT

Solid rear doors

Solid front doors

CR

AC

RackSingle rack

Multiple racks

Optional front containment

Figure 9 High-density pod with rack containment plus optional front containment



Selection criteria

No containment

Hot aisle containment

Rack air containment Comments

Minimize footprint

Good Good Moderate to poor

• NO containment and HOT AISLE containment provide minimum row spacing • RACK air containment adds 8 inches to the depth of

the rack but may be acceptable in consolidation applications • Front AND rear containment adds 16 inches to the

depth of the rack – should be weighed against available floor space

Ease of change management

Good Moderate to poor Moderate to poor • Taking racks in and out of an existing row is more

difficult when containment systems constrain the rack with hardware, especially with front containment

Minimize energy consumption

Moderate Good Good • NO containment layout is closely linked to the existing

data center layout which could increase the number of row-based units

Ease of redundancy

Moderate Good Moderate to poor

• HOT AISLE containment row-based CRAC positions are independent of redundancy • More row-based CRACs needed to maintain

redundancy in rack containment

Minimize # of row-based CRACs (particularly at low density)

Poor to moderate Good Moderate to good

• RACK air containment and RACK air containment with front containment may be limited since not all rack air can be shared among all row-based coolers as with HOT AISLE containment • NO containment depends heavily on rack power

density where high densities require less row-based coolers • RACK air containment and rack air containment with

front containment highly-influenced by redundancy (more coolers needed)

Sound attenuation

Poor Moderate to poor Good

• Poor to moderate with RACK air containment only • Good when using RACK air containment with front

containment • Will reduce the decibel level of the cooling equipment

but will not completely eliminate the noise

Installation in thermally unstable or non-data center space

Poor Poor Good

• Poor to moderate with RACK air containment only • Good when using RACK air containment with front

containment • Examples include wiring closets, offices, and

commercial spaces

Cost Dependent upon variables such as rack power density and number racks

• Although the hot aisle containment has additional panels that increase cost, it will require fewer row-based coolers than no containment, particularly at lower rack power densities

Table 1 Comparison of pod containment methods



The decision on whether to move forward with deployment of a high-density pod should also consider the following benefits: • Standardization of design elements

• Compatibility with any data center, new or existing

• Configurability with dedicated UPS and power distribution

• Configurability with any level of redundancy

• Configurability with any number of IT racks

Standardized design elements In order for high-density pods to provide predictable performance they must include standard design elements. This includes components such as air conditioners, power distribution, UPS, and racks. In addition, standard dimensions play a key role in predictably isolating hot and cold air flows. Standard dimensions include hot / cold aisle widths, rack height, and standard (short) airflow travel distances. Modularity is also a benefit of standardization and allows high-density pods to be quickly deployed, altered over time, and even moved to another data center. Standardized compo-nents and dimensions greatly simplify the design process. These pre-designed standard solutions may even be re-ordered for other data centers. Data center personnel can also leverage standardization by deploying predictable capacity and change management software that maintains the peak performance of the high-density pod (this is discussed later). For more information on standardization see White Paper 116, Standardization and Modularity in Network-Critical Physical Infrastructure. Compatible with any data center, new or existing High-density pods are modular and independent of room-based cooling architectures and existing UPS architectures. Therefore, few constraints exist to prevent their deployment in new or existing data centers. Sufficient floor space must be available and the floor must have enough weight-bearing capacity. All other aspects of a standardized high-density pod are replicable in multiple types of data centers. Configurable with dedicated UPS and power distribution The architecture of the high-density pod allows for deployment of pod-specific UPS and PDU configurations in cases where the existing data center UPS is at capacity or is being phased out due to end-of-life. These systems are rack-based and designed to be modular and scalable. Configurable with any level of redundancy Redundancy levels vary depending upon the criticality of the IT assets. Traditional data center design is such that the entire physical infrastructure is built to satisfy the redundancy requirements of the most critical set of assets. This type of design is extremely expensive both from a capital cost and operational cost perspective. A much more cost-effective design is to provide redundant power and cooling only where and when required. High-density pods allow for this targeted redundancy / availability approach by including redundant power and cooling modules when appropriate. Note that the core infrastructure such as chilled water

Additional high-density pod benefits

Standardization and Modularity in Network-Critical Physical Infrastructure





piping and electrical service entrance must be designed and built on day one with the highest redundancy level required. Configurable with any number of IT racks High-density pods are scalable in that they accommodate the number of IT racks required at a specific power density. These pods can range in size from a single IT rack to 20 or more racks depending on local codes. Combining these characteristics results in a highly flexible, high-density solution that can extend the life of a legacy data center and postpone the capital outlay required for building a new one.

The data center owner has two options for the deployment of high-density pods: in-house deployment or vendor-assisted deployment. In both cases a solid project plan is required. More specific information regarding data center projects and system planning is available in white papers 140, Data Center Projects: Standardized Process and 142, Data Center Projects: System Planning.

Attribute Traditional approach

Pod approach Comments

Positioning the data center as a source of corporate competitive advantage

Difficult Easier Simple economics – cost of doing

business is lower per unit of computational output

Just-in-time IT deployments

Very difficult Easy

Deployments are highly dependent on modular and predictable power and cooling which affects manage-ment and ability to quickly deploy

Predictability of performance

High Low Strongly linked to data center infrastructure efficiency

Likelihood of hot spots High Very low Management applications insure

optimal placement of equipment in pods to prevent hot spots

Cooling efficiency Poor Excellent

Room based cooling units are oversized to overcome under-floor

obstacles, distance, air mixing, demand fighting, etc.

Ability to plan Poor Excellent Standardization / predictability

facilitate “what-if” scenarios before moves, adds, and changes

Table 2 Deploying high-density equipment: traditional vs. pod approach

In-house vs. vendor-assisted deployment



Table 3 High-density pod compo-nents under various constraints

In-house deployment IT managers can easily deploy smaller sized pods or smaller data centers (less than 20 racks) with no previous experience. A worksheet and checklist is provided in Appendix A. This worksheet can serve as a helpful guide and facilitates the collection of information required to specify and deploy a high-density pod. The worksheet assumes the project owner has knowledge of the IT equipment associated with the planned high-density pod (e.g. total power requirements, plug requirements, rack U-height requirements and communications cabling requirements). If the worksheet is properly filled out, an educated decision can be made on which pod containment method to choose. APC TradeOff Tool™, Data Center InRow™ Containment Selector, (see Figure 10) can help select the most appropriate pod containment method. The results generated by the tool are based on typical scenarios and in some cases the recommended containment option may differ from the actual final design. Once a containment type is chosen, a decision must be made on which components the pod will include. The worksheet helps data center staff determine whether to include a dedicated UPS, PDU, or chiller. In some cases, certain preferences and constraints dictate which components are included in a pod and which are not. Table 3 provides a list of possible constraints that could affect the ultimate configuration of the high-density pod.

Constraint High-density pod requirement

None Racks and row-based cooling units

No spare power distribution positions

Racks, row-based cooling unites, and row-based power distribution unit (PDU)

No spare power capacity on existing UPS system

Racks, row-based cooling units, row-based UPS system, and row-based PDU

No spare cooling capacity on existing chiller

Racks, row-based cooling units, and packaged chiller

No spare power or cooling capacity on existing UPS and chiller

Racks, row-based cooling units, row-based UPS system, and packaged chiller

No spare power or cooling capacity on existing UPS and chiller or spare power distribution positions

Racks, row-based cooling units, row-based UPS system, packaged chiller, and row-based PDU

Related resource APC TradeOff ToolTM

Data Center InRowTM Containment Selector

Data Center Projects: System Planning


Data Center Projects: Standardized Process




http://www.apc.com/tool/?tt=10



Figure 10 Interactive tool for containment method selection

Figure 11 Packaged standalone high-density pod

Even with the constraint of no spare UPS, chiller, or power distribution capacity, it is still possible to extend the life of an existing data center by installing a high-density pod with its own dedicated power and cooling resources. For example, the high-density pod in Figure 11 includes its own chiller plant, UPS, and power distribution. It is assumed that the data center’s electrical service entrance has sufficient spare capacity to supply power to this packaged solution. In cases where a data center has run out of spare electrical service capacity, a decision must be made to install an additional utility feed(s) or build a new data center. Other factors beyond the scope of this paper such as available floor space, virtualiza-tion potential, business objectives, leasing contracts, and future growth plans factor into the buy-or-build decision.

Packaged chiller

Cooling distribution unit

http://www.apc.com/tool/?tt=10



From the time a need for a high-density pod is identified, IT and facilities personnel can expect to populate the racks in a given pod in one to three months, assuming the required budget is approved. However, internal company processes may extend the proposed timeline. Vendor-assisted deployment Although it is possible for data center staff to deploy high-density pods without outside assistance, projects involving data centers with 20 or more racks can be considerably more complex. In such cases consultation with design experts and project managers is recom-mended. Vendor-assisted deployment usually begins with an assessment of the existing data center or the design plans for a new data center. In either case an assessment provides the design experts with valuable information, including preferences and constraints, which allows optimum design decisions. Assessments help answer questions such as: • Can an existing row be retrofit with row-based air conditioners to avoid downtime?

• If spare chilled water capacity is unavailable should a self-contained air conditioning unit be used as opposed to a packaged chiller?

• What steps can be taken to increase the speed of deployment of a future high-density pod?

An effective assessment (such as Schneider Electric’s Blade Server Readiness Assessment) measures spare bulk power and cooling capacity as well as spare distribution capacity. Bulk cooling capacity is measured at the chiller while the distribution capacity is measured at the CRAH units on the data center floor. This data provides an estimate of cooling capacity and compares constraints against current and future requirements. Ultimately this will help answer the question, “When will I run out of cooling capacity and require a high-density pod?” After measuring and analyzing the data, a plan is created to meet future high-density needs. In the end, an effective design plan for mixed-density data centers should incorporate power, cooling and floor space utilization efficiency. An effective design plan allows a data center to use up its power, cooling, and space resources all at the same point in the future, thereby avoiding stranded resources. The architecture of row-based cooling makes real-time modeling of cooling performance possible. Design tools can configure racks, row-based air conditioners, UPS, and power distribution based on high-density pod specifications such as average and peak power density per rack, containment, redundancy, and plug types. Once a high-density pod is deployed, real-time planning and management tools allow IT personnel to maintain predicta-ble operation even after moves, adds, and changes take place. Examples of appropriate design and planning tools include InfraStruXure Designer and APC’s Capacity and Change Manager. For more information on management and its critical role in predictable perfor-mance, see White Paper 150, Power and Cooling Capacity Management for Data Centers.

Real-time management of high-density pods

Power and Cooling Capacity Management for Data Centers





In the past it was a major challenge for IT personnel to successfully deploy a mix of high-density and low-density equipment in the same data center space. Traditional data centers were specified to cool a uniform rack power density and were not capable of predictably cooling a large number of high-density racks. Now architectures such as row-based cooling allow for the rapid deployment of high-density pods within an existing or new low-density data center. Modular row-oriented power and cooling can be added where and when high-density racks are required, without any negative effect on the existing room-level infrastructure. In combination with capacity and change management systems, pods offer a high-density deployment solution capable of maintaining predictable operation even after moves, adds, and changes.

Conclusion

Neil Rasmussen is a Senior VP of Innovation for Schneider Electric. He establishes the technology direction for the world’s largest R&D budget devoted to power, cooling, and rack infrastructure for critical networks. Neil holds 19 patents related to high-efficiency and high-density data center power and cooling infrastructure, and has published over 50 white papers related to power and cooling systems, many published in more than 10 languages, most recently with a focus on the improvement of energy efficiency. He is an internationally recognized keynote speaker on the subject of high-efficiency data centers. Neil is currently working to advance the science of high-efficiency, high-density, scalable data center infrastructure solutions and is a principal architect of the APC InfraStruXure system. Prior to founding APC in 1981, Neil received his bachelors and masters degrees from MIT in electrical engineering, where he did his thesis on the analysis of a 200MW power supply for a tokamak fusion reactor. From 1979 to 1981 he worked at MIT Lincoln Laboratories on flywheel energy storage systems and solar electric power systems. Victor Avelar is a Senior Research Analyst at Schneider Electric. He is responsible for data center design and operations research, and consults with clients on risk assessment and design practices to optimize the availability and efficiency of their data center environments. Victor holds a Bachelor’s degree in Mechanical Engineering from Rensselaer Polytechnic Institute and an MBA from Babson College. He is a member of AFCOM and the American Society for Quality.

About the author



Hot-Aisle vs. Cold-Aisle Containment for Data Centers White Paper 135

Cooling Strategies for Ultra-High Density Rack and Blade Servers White Paper 46

The Advantages of Row and Rack-Oriented Cooling Architectures for Data Centers White Paper 130

Standardization and Modularity in Network- Critical Physical Infrastructure White Paper 116 Data Center Projects:Standardized Process White Paper 140

Data Center Projects: System Planning White Paper 142

Power and Cooling Capacity Management for Data Centers White Paper 150

Improving Rack Cooling Performance Using Airflow ManagementTM Blanking Panels White Paper 44

Airflow Uniformity Through Perforated Tiles in a Raised-Floor Data Center White Paper 121

Resources Click on icon to link to resource

Browse all white papers whitepapers.apc.com

tools.apc.com

Browse all TradeOff Tools™

For feedback and comments about the content of this white paper: Data Center Science Center [email protected] If you are a customer and have questions specific to your data center project: Contact your Schneider Electric representative

Contact us










http://whitepapers.apc.com/

http://tools.apc.com/



Data / input Value Comments

The

pod

1 Criticality level:

1, 2, 3, or 4

Goal for the availability and reliability of the pod, consistent with the business mission. See White

Paper 122 for guidance and choosing an appropri-ate criticality level.

2 Average rack power density (kW) Average IT load per rack. (The pod’s cooling will be designed to handle this.)

3 Peak rack power density Maximum anticipated IT load in any rack of the pod. (The pod’s cooling will be designed to handle this.)

4 Is there sufficient service entrance

capacity to support this high-density pod? (Yes / No)

Electrical service entrance capacity must be able to

support the incremental power load added by the pod (IT load plus power/cooling infrastructure).

The

room

5 Type of data center floor: raised floor

vs. hard floor

6

What floor-to-ceiling height is available for the pod’s equipment, taking service

clearances into account? (indicate ft. or m.)

The height available for proposed and future equipment, taking into consideration all applicable

service clearances per local jurisdiction. For example, sprinklers will affect available height.

Pow

er

7 Will pod include a separate dedicated

UPS? If no, skip to item 12.

8 Whit is available source input voltage to

the UPS(s) or PDU(s)?

9 How much current is available from the subpanel that will power the UPS(s) or

PDU(s)?

Total spare current of the subpanel feeding the room – must be shared with cooling equipment in item 21. Electrician is best source for information.

10 How many 3-pole breaker positions are

available in the panel(s)? Total number of spare 3-pole positions available to

be used by UPS(s) and PUD(s). Electrician is best source of this information.

11 What is preferred UPS runtime?

(minutes) See White Paper 52 for guidance

12 Total spare capacity of all existing UPS system(s) dedicated to the data center

This will determine if there is sufficient existing UPS capacity to support the new high-density pod,

taking into consideration desired redundancy and distribution.

13 How will the electrical distribution

cables be routed from the PDU(s) to the racks? (overhead or underfloor)

Appendix A: Worksheet and checklist for deployment of a high-density pod




Coo

ling

14 Which method will this high-density pod

use? No containment, hot aisle con-tainment, or rack containment?

15 What existing heat rejection methods

are available at the site? Chilled water, glycol, refrigerant, water cooled?

Identifies the types of heat rejection systems that

are available at the site. This will help in designing a high-density pod with a compatible cooling system.

16 What is the total “sensible capacity” (kW) of the existing cooling system?

The total cooling capacity (in kW) available from the existing cooling system. For chilled water

systems, this will be the capacity of the chiller plant. For DX systems, this will be the total of all the CRAC

units.

17 What is the spare “sensible capacity” of the existing chilled water system? (kW) To be filled out if chilled water spare capacity will

be used for this high-density pod.

18 How is cooling system piping routed?

Overhead or underfloor?

Identifies how the following are routed: DX – glycol, condenser water, humidification and

condensate lines Chilled water – supply/return piping

19 How will the chilled water piping be

routed to the new cooling units? Overhead or underfloor?

Routing for refrigerant, humidification, and condensate lines

20 What is the source input voltage to the

new CRAC / CRAH unit(s)? (volts) If no, skip to item 12.

21 How much current is available to power the new CRAC / CRAH unit(s)? (amps)

22 Does the cooling solution require both critical and non-critical power inputs? If NO, skip next two items.

23 What voltage feeds the critical power

input of the cooling unit? Voltage for power to fans and controls

24 What voltage feeds the non-critcal

power input of the cooling unit? Voltage for power to compressor (DX only), humidifier, and pump




Mon

itor

ing

/ man

agem

ent

25 What type of physical security system is

required for the high-density pod? Door card, cameras, motion detectors?

26 What building management system (BMS) does the existing data center

use? (Name of system or none used)

27 What network management system

does the existing data center use? (Name of system or none used)

28 What is the preferred level of instru-

mentation? (typical or full) Identifies the preferred instrumentation level for

the high-density pod, using various sensors such as temperature, humidity, water and motion.

29 How is the structured cabling within the

data center routed? Overhead or underfloor?

“Structured cabling” refers to networking cables connecting infrastructure equipment.

Deploying High-Density Pods in a Low-Density Data … High-Density Pods in a Low-Density Data Center...

Documents

Transcript of Deploying High-Density Pods in a Low-Density Data … High-Density Pods in a Low-Density Data Center...