DATA CENTER TRENDS TODAY & TOMORROW

www.eypmcfinc.com Follow us on:

DATA CENTER TRENDSTODAY & TOMORROW

STEVEN SHAPIRO, PE ATD

https://www.facebook.com/eypmcf/

https://twitter.com/eypmcf

https://www.linkedin.com/company/eypmcf/

MEET YOUR PRESENTERS

Steve ShapiroPartner, EYP Mission Critical Facilities

Jeff Margolin

Principal, Ramboll (Moderator)


AGENDA

• IT Trends

• Standards/Guidelines/Codes

• Cooling Design Trends

• Power Design Trends

https://www.eypmcfinc.com/




Follow us on:

DATA CENTERIT TRENDS





IT TRENDS

• Density

• Density vs. reliability profile

• Physical footprint






IT EQUIPMENT TRENDS

Manufacturers of IT equipment recognize this demand for increased power.

• Footprint reduction• Decreased outlet temperature• State-of-the-art microchip technology• Open compute architecture

As design engineers, we must support this shift in paradigm…Both electrically and mechanically!






DENSITY

• For every 18F rise in temperature hardware reliability decreased by 50%

• Footprint reduction = 30% per year

• Since 2000, power consumption for chips doubled

• 17% annual increase in load density in past 10 years






IT TRENDS

Chip manufacturers are investigating:

• New sink architectures

• Water cooled sinks

• Larger cooling fans (up to 70 cfm per server)

ADVANCED COOLING TRENDS






PHYSICAL FOOTPRINT

• The rack is getting crowded

• Multiple PDUs, copper and fiber connections are making wire management a mess

• Power to the rack has many options

• Scalability is an issue

• Allocation of power is an issue

• Power metering is an issue

• How do the power cords really get utilized with the new equipment?

• How do we keep the SLAs?






DENSITY AND RELIABILITY PROFILE

300

TIE

RIV

III

II

25020015010050

W/SF

ENTERPRISE

Hyperscale

COLO

Design Criteria

I

OLD MAINFRAME






DENSITY AND RELIABILITY PROFILE

• Enterprise vs. Colo vs. public cloud

• Hybrid IT strategy – on premises/private cloud

• Multi-cloud – private and public cloud (AWS/AZURE/Google)





Follow us on:

DATA CENTER STANDARDS/GUIDELINES/CODES





GENERAL STANDARDS/GUIDELINES/CODES

• Uptime Institute Tier Certification: Tier I to IV• ANSI/BICSI 002-2014 – Class F0 to F4• TIA 942 – Tier 1-4• ICREA• UL• EN 50600 series 1, 2-1 to 2-6 – Availability Class 1-4• EN 50173-5• ISO/IEC 24764• HIPPA – Health Insurance Portability and Accountability Act• SOX – Sarbanes Oxley – 2002• SAS 70 – Type I or II• Gramm-Leach Bliley Act (GLBA)• FedRamp & NIST

UPTIME/RELIABILITY DESIGN AND INFRASTRUCTURE STANDARDS






GENERAL GUIDELINES/REQUIREMENTS

• Uptime Institute: Operational Sustainability

• ISO 9000 – Quality System

• ISO14000 – Environmental Management System

• IS0 27001 – Information Security

• PCI – Payment Card Industry Security Standard

• SAS70 & ISAE 3402 or SSAE16 (USA) – Assurance controls

• AMS-IX – Data Centre Business Continuity Standard

• EN50600-2-6 Management and Operational Information

OPERATIONS STANDARDS







Energy Efficiency Standards

• LEED

• Energy Star

• Green Globes

• CEEDA

Sustainability Standards?

UPTIME/RELIABILITY DESIGN AND INFRASTRUCTURE STANDARDS







NFPA – National Fire Protection Association

• National Electrical Code – NFPA70

• NFPA 855 Standard For The Installation Of Stationary Energy Storage Systems

• NFPA 70E – Standard for Electrical Safety in the Workplace

• NFPA 72 – National Fire Alarm Code

IBC – International Building Codes

CODES





Follow us on:

DATA CENTER MECHANICAL TRENDS





Cooling Evolution/ Trends

Indirect / Direct

Evaporative Systems

Chip / Server Cooling

Immersion Cooling

Kyoto Cooling






COOLING DESIGN TRENDS

Fighting density and increasing energy efficiency

• In the data center

• For the data center• Systems selection process

• Economization alternatives






MECHANICAL TRENDS

Customer trends require a change in the way mechanical design is approached.

Electrical Engineers must adapt to these changes as well!

• High density IT racks require intensive cooling in localized areas

• Broader acceptable temperature ranges at the rack level

• Concurrent maintainability requirements mean multiple power feeds






MECHANICAL TRENDS

Packaged units are on the rise

• Water reduction techniques are being implemented

Free air cooling

• Direct – outside air does enter the data center

• Indirect – outside air does not enter the data center

New innovation – Kyoto Wheel






FOR THE DATA CENTERSYSTEM SELECTION PROCESS






FOR THE DATA CENTER

• Chiller/cooling tower

• Air cooled chiller

• Adiabatic cooling/swamp cooler (roof, ground, CRAH)

• Outside air

• Air to air (Munthers, Schneider, Hunt Aire, Kyoto)

• Water side economizer • (Heat exchanger)

• DX backup/full-partial

• Drycoolers

SYSTEM ALTERNATIVES






FOR THE DATA CENTERSYSTEM ALTERNATIVES

Direct Air Economizer System

Outside air does not enter

the data center

Indirect Air Economizer System

Outside air does enter

the data center






FOR THE DATA CENTERSYSTEM ALTERNATIVES

Direct Evaporative Air Economizer

“Swamp” cooler without outside air

entering the data center

Indirect Evaporative Air Economizer

“Swamp” cooler with outside air

entering the data center






FOR THE DATA CENTERSYSTEM ALTERNATIVES – KYOTO WHEEL

Indirect Air-side Economizer






IN THE DATA CENTERFAN POWERED CABINET EXAMPLE

Cabinet Fans

Hot Air Plenum

Conditioned Air

is drawn through the

front of the cabinet

Hot Air

is discharged out of

the top via ductwork

Raised Floor






IN THE DATA CENTERWATER COOLED CABINET EXAMPLE

Conditioned Air



Fan Coil

on rear door

cools servers

Cabinet Fans

Cooling Coil

Supply Return Piping Raised Floor

Discharge Air

High Pressure

flex hose and quick-connect

fittings to chilled water mains






IN THE DATA CENTERHOT AISLE COLD AISLE CONFIGURATION

Conditioned Air



Hot Air

is discharged out of

the back of the cabinet

Conditioned Air



Return Air

Cold Hot ColdAisle Aisle Aisle






IN THE DATA CENTER

• Immersion cooling – bathtub

• Liquid cooling – to the server

• Liquid cooling – to the chip

• Rear door cooling

• Issues in existing facilities

• Issues with space vs. power requirements

LIQUID COOLING






FOR THE DATA CENTERRaised Floor No Raised Floor

Free movement of cabinets Cost savings

Flexible placement of venting tiles Overhead cable plant

Air flow control – easy to deal with high density cabinets

Increased floor to ceiling height

Cable concealment No weight concerns

EMC shielding/SRG No under floor fire protection

Concealment of plumbing Appearance?

Appearance Air control – may need containment

Additional cost Containment – added cost

Reduced floor to ceiling height

Underfloor fire detection/protection required

Clogged air paths

Issues with removal of too many tiles at a time

Not durable - damaged flooring over time






COOLING TECHNOLOGIES

• Data centers must meet an ISO Class 8 Clean Room Standard

• Containment

• Chimneys

• Cold aisle

• Hot aisle

• CFD analysis






MECHANICAL SYSTEMS

• CBRN – chemical, biological, radiological, nuclear filtration/threat

• Water storage

• Redundant piping, loop with dual risers, redundant coils

• Water treatment

• Fuel storage – 48, 72, 96 hours, above or below, day tanks, belly tanks

• Fuel treatment – polishing

• Fuel delivery/testing on delivery

• Fuel sampling






ELECTRO-MECHANICAL TRENDS

Power Usage Effectiveness (PUE) – the litmus test between the trades

How do we calculate PUE?





Follow us on:

DATA CENTER ELECTRICAL TRENDS





ELECTRICAL TRENDS

• Increased critical power requirements leads to more electrical design options

• Reliability and availability

• Maintainability

• Modularization and flexibility

• Energy efficiency

• Infrastructure utilization






ELECTRICAL TRENDS

• Green and energy savings – sustainability

• Solar

• Fuel cells

• Micro-turbines

• Wind

• Distributed generation and load curtailment

• Emissions – prime/standby/scrubbers

• Demand peak shaving

• Lighting controls






ELECTRICAL TRENDS

• Energy storage

• Lithium-ion batteries

• Sodium/nickel zinc, etc.

• Natural gas/hydrogen generation

• Compressed air storage

• Hydraulic storage

• Microgrid

• Other renewables






ELECTRICAL TRENDS

• The desire to scale the facility as the business grows allows the customer to minimize up front costs

• Initial investment

• We as engineers need to allow for this growthpotential when considering topology and floor plan layouts

• Many vendors have added self contained pods to their lines which cater to this need

MODULARIZATION & SCALABILITY






ELECTRICAL TRENDS

UPS Manufacturers recognize these customer driven demands and have implemented them into their latest products

• Distributed paralleling are trending up!

• Central static switches are getting unpopular!

• Higher infrastructure cost

• Modularization/scalability

• Transformer free modules are trending up!

• Increased efficiency with mitigated heat loss

• Less cooling required!

• Eco mode

• Running the module on bypass during normal voltage conditions

• Increase in efficiency!






ELECTRICAL TRENDS

Efficiency is the key to success… and UPS manufacturers know this!

• Silicon Controlled Rectifiers (SCR) based modules ~80% efficient

• Transistors (IGBT) replace SCR ~92% efficient

• Transformer free ~97% efficient

• Eco mode introduced ~99% efficient






ELECTRICAL TRENDS

400V distribution system

Eliminate the need for PDUs and deliver critical power directly from the UPS!

• Still maintain use of standard server equipment

• Reduces HVAC requirements

• Increases overall electrical efficiency

• Decreases footprint requirements for infrastructure

• Reduces maintenance costs

THINKING OUTSIDE THE “UPS BOX”






ELECTRICAL TRENDS

DC distribution system!• Still maintain use of standard server equipment

• More efficient than AC

• Less cable loses over long distances

• Lower cost

• No conversion losses

• Compatible with many alternative energy sources

THINKING OUTSIDE THE “UPS BOX”

Doesn’t the power ultimately get converted into DC at the server level?






ELECTRICAL TRENDS

Unfortunately, many customers are reluctant to try such radical ideas…

…so let’s stick with the UPS.






ELECTRICAL TRENDSUPTIME INSTITUTE TIER CERTIFICATIONS

Tier I Tier II Tier III Tier IV

Active capacity components to

support the IT load

N N+1 N+1 N after any failure

Distribution Paths 1 1 1 active

1 alternate

2 simultaneously

active

Concurrently Maintainable No No Yes Yes

Fault Tolerant No No No Yes

Compartmentalization No No No Yes

Continues Cooling No No No Yes

What are the customer’s goals?






CONCURRENT MAINTAINABILITY

• “Change tires, transmission and engine at 60mph… and don’t lose speed.”

• No IT downtime for preventative and corrective maintenance

• Scalability

GOALS






ELECTRICAL TRENDSTHE CUSTOMER’S GOALS DRIVE CRITICAL TOPOLOGY

Traditional:• N• N+1

N+N






RELIABILITY/AVAILABILITYTYPICAL UPS CONFIGURATIONS

PARALLEL

REDUNDANT

2N







DISTRIBUTED

REDUNDANT

3 to Make 2







BLOCK REDUNDANT 3+1






RELIABILITY/AVAILABILITY

Description of RBDMTBF

(Hours)

Inherent Availability

(Ai)

Probability of Failure

in 5 years

N + 1 UPS system – dual cord loads 32,509 0.99981626 58.16%

Distributed Redundant (3 to make 2) UPS system –

dual cord loads161,646 0.99997994 7.43%

2N UPS system – dual cord loads 214,182 0.99998723 6.56%

2(N + 1) UPS system – dual cord loads 305,251 0.9999868 6.49%

Utility and N + 1 UPS system, ASTSs – dual cord

loads65,056 0.99999821 8.02%

Block Reserve (2-3) UPS System, ASTSs – dual

cord loads257,459 0.99999058 2.58%

Distributed Redundant (3 to make 2) UPS system,

ASTSs – dual cord loads256,674 0.99999046 2.72%

2N UPS system, ASTSs – dual cord loads 445,691 0.99999845 1.12%

2(N + 1) UPS system, ASTSs – dual cord loads 989,960 0.99999839 0.88%






RELIABILITY/AVAILABILITY

• Load density (the major issue)

• 99.9999% availability (“Six Sigma”)

• 2(N+1), 2N, N+2 redundancy not so common

• N or (N+1) systems not good enough

• Fault tolerant configurations – eliminate SPOFs

• Full concurrent maintenance






THE ECONOMICAL SOLUTION

• Dual/diverse utility services

• Dual active distribution paths

• N+2 generator plant

• “Self-healing” automation and control

• Physical separation/compartmentalization

• Distributed redundant UPS

• Dual electrical cords to IT cabinets






KEY TAKEAWAYS

1. IT trends

2. Codes and standards

3. Mechanical systems trends

4. Electrical systems trends






THANK YOU AND PLEASE FEEL FREE TO CONTACT ME

Steven Shapiro, PE, ATD914.420.3213

[email protected]

http://www.linkedin.com/in/stevenshapirope

@stevenshapirope





http://www.linkedin.com/in/stevenshapirope

UPCOMING WEBINARSamericas.ramboll.com/webinar/data-centers

Trade-offs and Co-benefits: Balancing Long-Term Sustainability Goals with Time-to-Market NeedsMarch 31 – Amy Malick, Ramboll

Integration – Toward Net PositiveApril 21 – Expert panel from Ramboll, EYP Mission Critical Facilities and PSC

Questions?Steve Shapiro | Partner | [email protected]

DATA CENTER TRENDS TODAY & TOMORROW

Documents

Transcript of DATA CENTER TRENDS TODAY & TOMORROW