New technologies and architectures for efficient data centers 2015 Report by Yole Developpement

From Technologies to Market

Sample

From Technologies to Market

© 2015

New technologies & architectures

for efficient Data Centers

2

TABLE OF CONTENTS

• Report Objectives 4

• What Is In This Report? 5

• Glossary 7

• Executive Summary 8

• Today’s Challenges on New Generation Data Centers 29

Big Metrics 34

Technological Breakthroughs 42

• General Aspects Of Data Centers 54

Power Consumption – Energy Efficiency Issues 61

• Supply Chain 74

• Architectures & Power Systems 81

Traditional Architectures 89

Modular & Container Architectures 99

• Technical Solutions & Technological Breakthroughs 116

DC Grid 118

Servers 135

o Emerging NVM 141

Silicon Photonics 154

Uninterruptible Power Supply 170

Wide Band Gap Devices 179

Cooling Systems 187

• Conclusions 194

• Latest News 198

• Yole Presentation 202

©2015 | www.yole.fr | New Technologies and Architectures for Efficient Data Centers

http://www.yole.fr/

http://www.yole.fr/

3

COMPANIES CITED IN THE REPORT

Cisco, Arista Networks, HP, Dell, Vmware, Brocade, Lenovo, Huawei, Extreme Networks, Avaya, Toshiba, EPC, Samsung, SanDisk, Micron, Sony, SK Hynix, Violin memory, STEC, Tms, Fusion-io, Eaton Corp., Emerson Electric, Schneider Electric, General Electric,

Belkin Industries, Socomec, Newave, Emerson Network Power, Stulz, Uniflair, GEA Group, APC, Air Data, Airedale, RC Group, Intel, IBM, Luxtera, Mellanox, Molex, SGI, Sun Microsystems, Lampertz, Bull, ActivePower, Verari Forest, Delta, ABB, Digital Really, QTS Data Centers, Sabey Data Centers, DuPont Fabros, Equinix, NTT, Facebook, Google, Ebay, Amazon, Microsoft, and more!

http://www.yole.fr/

http://www.yole.fr/

4

MAIN 4 CHARACTERISTICS

• High concentration of heat

• All the concentrated electronics generate huge amount of heat in a closed space

• Large cooling systems have to be installed

• Huge amounts of data

• Increasing need for bandwidth

• Interconnexions between servers are done thanks to being at the same physical site

• Data Centers need to be close to big urban areas

• Very high electricity consumption• A proximity of a point access to very high

electrical power is required

• Multi-sourcing is obligatory for safety reasons

• Strong trends towards the use of renewableelectricity sources (hydro, PV, wind…)

• Ways to reduce the consumption are activelyresearched

• Requirements on extremely high (24/7) data availabilityand security

• Solutions implemented to ensure “ALWAYS-ON state” - uninterruptedpower supply, redundancy - System resilient to failure and enabling rapidrecovery after failure

• System protected against human and natural detrimental influence

©2015 | www.yole.fr | New technologies & architectures for efficient data centers

http://www.yole.fr/

http://www.yole.fr/

5

POWER REQUIREMENTS

Power needed by the IT equipment


http://www.yole.fr/

http://www.yole.fr/

6

POWER REQUIREMENTS OF A DATA CENTER

In order to feed the IT equipment, more than double of power is needed

• In a traditional Data Center, about50% of the power consumption isused for something else than the ITloads.

• This means that to supply the ITequipment with a power of 60 kW, atleast 150 kW are required. (For a PUEof 2.5)

• Virtually, every watt expended in aserver room in terms of processingpower, power supplies, lighting, etc. isturned into heat.

• Thereby, huge amounts of heat have tobe dissipated in a relatively small andclose environment.

Traditional Data Centers are considered extremely inefficient, as in many cases less than half of the power is used for IT loads

PUE = 2.5


http://www.yole.fr/

http://www.yole.fr/

7

RENEWABLE ENERGY SOURCES

Different alternative power sources

• Several projects have been presented using renewable energies to power their data centers:

In order to reduce the electricity bill, Data Centers are incorporating additional alternative power sources

Geothermal Data Centers: A number of data

centers in the Midwest (US) are using geothermal

cooling in their data centers. Then there’s Iceland

and its plentiful supply of geothermal energy.

In the US, there is a potential of >2,000 GW on

geothermal energy

Waste Heat Reclamation: Most data centers are

already generating an energy source: the heat

emerging from the back of their server racks.

Here’s a look at some facilities that are reusing

this waste heat.

Wind-Powered Data Centers: Only a handful of

companies have implemented wind turbines in

working data centers, but new players have some

ambitious plans in this area.

Solar-Powered Data Centers: Emerson Network

Power, i/o Data Centers and AISO are among the

companies that have implemented on-site solar

solutions.

Waste heat from servers at the new Telecity Paris data center (left) is

being used to heat an on-site arboretum (right).


http://www.yole.fr/

http://www.yole.fr/

8

GLOBAL ENERGY CONSUMPTION

Evolution with a decreasing average PUE

• Each hypothesis fix new PUEs by region from the mentioned years on.

• Each new hypothesis also takes into account the previous hypothesis.

In the case that by 2019 PUEs would decrease considerably, with averages between 1.3-1.5, the global energy consumption will stay stable (below 400TWh) until 2020

-24%


http://www.yole.fr/

http://www.yole.fr/

9

ENTERPRISE STORAGE SUPPLY CHAIN

Emerging NVM could beused at both drive (HDD,SSD) and storage systemlevel for various cachememory applications. Earlyadopters are SSDmanufacturers whointegrate EverspinSTTMRAM for memorytype SCM applications. In asecond step, RRAM couldalso be used as storagetype SCM. Enterprisestorage end users will alsopush chip makers tointroduce new solutions inorder to follow data trafficexplosion.


http://www.yole.fr/

http://www.yole.fr/

10

F3 & F4 CLASSES

Multi-sourcing and circuit redundancy are the key features of the safest Data Centers

The two safest topologies – class 3 & class 4 – provides different power source access connections from the grid (multi-sourcing).

There are (at least) two completely independent power circuits, with component redundancy.

Class F4 Fault Tolerant

Class F3Not UPSs in every

distribution channel –Not a complete

redundancy

• The F3 class does not have a

complete circuit redundancy as

UPSs are used just in one of the two

independent circuits.

• The F4 topology guarantees that

every critical load is at least supplied

by two independent and equal

distribution circuits.


http://www.yole.fr/

http://www.yole.fr/

11

MARKET SPLIT OF ARCHITECTURES

Which topology is the most widespread?

• F0 and F1 architectures are the most expanded ones: >95%

• However, they represent server closet and small rooms which we do not consider as a Data Center.

• The real Data Center split is here below:

• Double power source,

UPS redundancy, but

not circuit redundancy

• Multi power sourcing,

circuit redundancy

Architecture class split by number of sites


http://www.yole.fr/

http://www.yole.fr/

12

REDUNDANCY

Redundancy increases the problem of overrating

• The load rate of the power converter affects its efficiency. In consequence, losses increase considerably when load decreases from 50 %.

• Servers rarely work at 100 % of their capacity. Furthermore redundancy divides the transmitted power of a component by the redundancy power factor.

• Therefore, components are rated for the 100 % of the capacity, when in reality they transfer less than 25 % of the load each (when double redundancy in circuit and component level).

Redundancy power factor = Component redundancy * Circuit redundancy

AC/DC rectifiers

Component (or converter)

level redundancy

System without

redundancy

Redundancy

Circuit level redundancy1st Converter 2nd Converter 2nd power distribution circuitWorking nearly at

100% of load rate

Each converter is

working at 25% of its

load rate


http://www.yole.fr/

http://www.yole.fr/

13

CONTAINERS

Modular “Plug & Play” solutions

• Rugged, portable and energy efficient containerized datacenters that provide plug & play datacentercapacity inside an ISO container in a fast and cost effective way.

All-in-one compact units providing a fully functional data center

AST Modular containers:- Up to 12x42 servers

- Up to 96 kW in power and cooling

- N+1 redundancy (power & cooling)

- 100kVA UPS

- 320sqft (29.7 m2)

Schneider Electric acquired

AST Modular in 2014

Huawei:- Up to 8x45 servers

- Up to 60 kW in IT

- Row-level air conditioner

horizontal air supply

- 120kVA UPS

- 320sqft (29.7 m2)

IDS1000A-40 container


http://www.yole.fr/

http://www.yole.fr/

14

CONTAINER DATA CENTER MANUFACTURERS

2014 market share

AST Modular was bought by Schneider Electric in the beginning of 2014

Other players are:

- Sun Microsystems

- Verari Forest

- Lampertz

- Bull

- ActivePower

- Cisco

- etc.


http://www.yole.fr/

http://www.yole.fr/

15

CONTAINER DATA CENTER MARKET

Market estimation in M$ for the period 2010-2020

• From 2014 on, this market will significantly

increase as many Data Center manufacturers

have seen the interest of using these modular

containers.

• Container based Data Centers

had an stable growth in the last

five years.

CAGR 15-20

+23.2%

CAGR 10-14

+11.7%


http://www.yole.fr/

http://www.yole.fr/

16

DC ARCHITECTURE

From AC to DC power distribution in Data Center

AC distribution: at least 3 sequential power

conversion stages involving a change from AC to

DC or vice versa

DC distribution removes 2 of the 3

sequential AC-DC/DC-AC power

conversion stages.

AC is 400 Vrms in Europe and 120 Vrms or 240 Vrms in the US

DC can be 48V, 326V or 380V depending on the region and the power load

POL: point-of-load

VRM: voltage regulation modules

Redundant

Redundant

A 10% to 20% energy saving on

the overall Data center is

expected by moving to DC

distribution


http://www.yole.fr/

http://www.yole.fr/

17

ARCHITECTURE COMPARISON

Design

Most innovative

concept are used in

installations for outdoor installations

(higher power and

less controlled

environment)

Innova

tio

ns

Tra

ditio

nal

arch

itect

ure

s

Traditional AC

systems

Row level DC

distribution

Facility level DC

distribution

Modular AC

systems Facility level DC

demonstrators

Increasing

environmental

constraints

In the short-mid

time future

DC solutions

AC solutions

Next coming

architectures


http://www.yole.fr/

http://www.yole.fr/

18

DC EQUIPMENT PLAYERS

• Several Data Center equipment suppliers are working on DC voltage solution, even though there is not a market for it yet.

• Special efforts are done at a reliable 400V DC breaker that would suit with Data Center safety & reliability requirements.

• So far, 48 VDC and 60VDC breakers are commercially available.

Players involved in the development of DC solutions:


http://www.yole.fr/

http://www.yole.fr/

19

FOCUS ON MEMORIES

Introduction of the typical memory types in data centers

Non-volatile memories stock data even if they are not supplied by electricity. Volatile memories are erase without electricity supply.

Working memory

DRAM

Speed

Gb capacity

> $1/Gb


http://www.yole.fr/

http://www.yole.fr/

20

RRAM CHIP MAXIMUM DENSITY ROADMAP

Stand-alone devices

RRAM will start being used in data centers from 2015 on as Storage Class Memory (SCM) 1 Gb

10 Mb

100 Mb

10 Gb

Chip Maximum

density

RRAM

High density

for

enterprise

storage

100 Gb

1 T b NAND

10 T b

3D NAND

Transition

Micron/sony

16 Gb

Adesto

64 Mb

RRAM

Low

density for

wearable &

industrial

Year of

commercial

sample

availability

Memory SCM

enterprise storage

applications

NAND

substitution in

data centers

Yole Développement © January 2015

Attacking the

data storage

application


http://www.yole.fr/

http://www.yole.fr/

21

EMERGING NVM MARKET FORECAST

By application from 2014–2020 (in $M)

CAGR 2014-2020

118%

85%

154%

35%

CAGR 2014-2020

118%

85%

154%

35%

CAGR 2014-2020

118%

85%

154%

35%


http://www.yole.fr/

http://www.yole.fr/

22

UPS MARKET

>200kVA UPS market dedicated to data centers


http://www.yole.fr/

http://www.yole.fr/

23

DATA CENTRE NETWORKING

Where optics is cool!

Just a few examples of photonicsolutions that willreducepower.

Router ASIC with integrated optics

(Compass-EOS) OOO routers (Calient)

AOC (Finisar)

Active Optic interconnect

(TE Connectivity)

Hybrid optical/electrical switching (UCSD)

Si photonics and closer integration

(Luxtera)


http://www.yole.fr/

http://www.yole.fr/

24

WHY SiC & GaN ADDED VALUE?

High electron mobility and high junction temperature are the key intrinsic characteristics.

High electron mobilityHigh Junction

Temperature

No recovery time

during switching

Low lossesless energy to dissipate

Fewer cooling

needs

System size and

weight

reduction

High switching

frequency

Smaller filters

and passives

Intrinsic

properties

Impact on

operation

Impact on

power module

Impact on

power system


http://www.yole.fr/

http://www.yole.fr/

New technologies and architectures for efficient data centers 2015 Report by Yole Developpement

Technology

Transcript of New technologies and architectures for efficient data centers 2015 Report by Yole Developpement